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Abstract 

This  document,  together  with  the  software  and  test  datasets  supplied 
and  installed  on  a  SUN  workstation  at  Phillips  Laboratory  and  on  the  Maui 
High  Performance  Computer  Center  SP2  computer,  are  the  completion  of 
the  contract  PIIN:F61708-95-C0010. 

1  Introduction 

This  document  is  the  Final  Report  for  the  work  undertaken  under  Contract 
F61708-95-C0010  between  BOARD  and  AEA  Technology,  Culham.  This  work 
is  a  successor  to  an  earlier  project  3-D  PIC  Software  for  MIMD  Computers, 
PIIN:F61708-9S-C0011,  and  the  material  presented  here  assumes  details  of  the 
algorithms  and  software  design  given  in  earlier  documents  issued  during  that 
predecessor  contract  [1,  2,  3,  4,  5,  6,  7,  8,  9,  10,  11,  12,  13,  14],  and  elsewhere 
[15,  16,  17,  18]. 

Attached  to  this  report  are  eight  Annexes  which  contain  results  of  the  par¬ 
allel  benchmarking,  further  algorithm  analysis,  test  examples  and  and  revised 
specifications.  These  Annexes  are: 

Annex  A: 

R  W  Hockney [19], 

LPM3  Benchmark  Results  on  the  MHPCC  IBM  SP2  Computer, 
Technical  Note  AEA/TYKB/28006/TN/1, 

Culham  Laboratory,  September  1995. 

This  Note  summarises  the  implementation  and  testing  of  the  new  mes¬ 
sage  passing  routines  and  compares  the  parallel  performance  of  different 
options  on  the  SP2  with  each  other  and  with  the  parallel  performance 
measurements  made  earlier  on  the  Intel  Paragon  at  Sandia. 

Annex  B: 

J  W  Eastwood,  W  Arter,  N  J  Brealey  and  R  W  Hockney [20], 
Demonstration  Calculations  of  HPM  Sources  using  3-D  Electromagnetic 
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PIC  Software, 

Technical  Note  AEA/TYKB/28006/TN/2, 

Culham  Laboratory,  September  1995. 

This  Note  describes  the  changes  made  to  produce  the  Version  2.10.00  of 
the  software  issued  with  this  report,  and  presents  test  results  obtained 
with  this  new  version. 

Annex  C: 

J  W  Eastwood,  W  Arter,  and  N  J  Brealey[21], 

3DPIC  Release  S.  10.00:  User  Guide, 

Technical  Note  AEA/TYKB/28006/TN/3, 

Culham  Laboratory,  September  1995. 

This  Note  contains  a  description  of  the  user  input  files  (.uif)s,  and  an 
introduction  to  running  the  preprocessor  PEGGIE,  the  simulation  code 
PIC3D,  postprocessors  TSDMERGE,  MPICTIM  and  DISPAN,  together  with 
the  GHOST  graphical  system. 

Annex  D: 

W  Arter [22], 

Release  2.10.00  of  the  PEGGIE  Preprocessor, 

Technical  Note  AEA/TYKB/28006/TN/4, 

Culham  Laboratory,  September  1995. 

This  Note  describes  the  PEGGIE  preprocessor  and  its  input  specification. 
The  concept  underlying  the  PEGGIE  (Parallel  Electromagnetic  General 
Geometry  Interface)  approach  to  grid  generation,  boundary  conditions, 
material  property  specification  and  diagnostics,  is  that  the  device  to  be 
modelled  and  the  diagnostic  lines,  surfaces  and  volumes  are  made  by 
joining  ‘parts’  together.  An  important  advantage  is  that  a  ‘part’  might 
be  output  of  another  geometry  package  (eg  [23]),  although  normally  it  will 
consist  of  a  subroutine  and  the  associated  variables.  The  Note  extends 
the  specification  give  in  Reference  [14],  and  supersedes  that  document. 

Annex  E: 

N  J  Brealey[24], 

MPICTIM  Release  2.10.00:  User  Guide, 

Technical  Note  AEA/TYKB/28006/TN/5, 

Culham  Laboratory,  September  1995. 

The  timeseries  output  files  (.tsd  files)  produced  by  PIC3D  diagnostics 
can  be  examined  on  a  UNIX  Workstation  using  the  OSF /Motif  GUI  tool 
MPICTIM.  This  Note  describes  how  to  use  Version  2.10.00  of  MPICTIM 
and  supersedes  Technical  Note  AEA/TYKB/31878/TN/10  [12]. 

Annex  F: 

J  W  Eastwood,  W  Arter,  N  J  Brealey  and  R  W  Hockney[18], 

Body  Fitted  Electromagnetic  PIC  Software  for  Use  on  Parallel  Computers, 
Comput  Phys  Commun  87(1995)155. 
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This  Annex  reproduces  the  paper  published  in  Computer  Physics  Com¬ 
munications  describing  the  P1C3D  algorithm  and  software. 

Annex  G: 

R  W  Hockney, 

MIMD  Version  PIC3D  Operating  Instructions  -  Maui  HPCC, 

August  1995. 

This  Annex  is  a  copy  of  the  “readme”  file  held  in  the  file 

"hockney/MAUI-SDPIC/README-MIMD 

on  the  MHPCC  SP2  computer. 

Annex  H: 

R  W  Hockney, 

LPMSBenchmarking, 

September  1995. 

This  Annex  is  a  copy  of  the  “readme”  file  held  in  the  file 
"ho  ckney /SP2/LPM3/ test /README-LPM3 
on  the  MHPCC  SP2  computer. 


2  Programme  of  Work 

The  objectives  of  the  work,  as  described  in  the  Proposal  [25]  and  restated  in 
the  contract  document  for  Contract  Number  F61708-95-C0010,  were: 

Subtask  1.1  Implement  and  test  new  message  passing  routines  for  use  on  the 
SP2. 

Subtask  1.2  Perform  LPM3  benchmarking  on  the  SP2. 

Subtask  1.3  Develop  PIC3D  kernel  for  the  SP2. 

Subtask  1.4  Prepare  final  report. 

These  Subtasks  are  discussed  in  the  following  Subsections  and  in  the  Annexes 
to  which  those  Subsections  refer. 

2.1  Subtask  1.1 

The  message  passing  software  was  developed  in  parallel  with  the  physics  ker¬ 
nel  of  the  main  simulation  program  PIC3D  (see  below).  The  testbed  for  the 
developing  message  passing  routines  was  the  benchmarking  version  of  the  code 
(LPM3).  LPM3  is  a  fully  functional  3-D  code  with  the  same  data  structures  and 
main  calculation  routines  as  PIC3D,  but  with  limited  physics  and  restricted  ge¬ 
ometric  capabilities. 

Two  different  versions  of  LPM3  were  developed;  one  using  a  ‘per-patch’  mes¬ 
sage  data  exchange,  and  one  using  a  ‘per-node’  exchange.  Both  cases  were  im¬ 
plemented  using  four  different  communications  interfaces  on  the  SP2  -  PVM3ip, 
PVM3sw,  PVMe  and  MPL  (cf  Annex  A). 

The  source  for  the  LPM3  benchmark  code  is  installed  on  the  SP2  in  directory 
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~hockney/SP2/LPM3/pvm3 

Test  benchmark  runs  should  be  carried  out  in  directory 
"hockney/SP2/LPM3/test 

which  contains  the  input  data  files  for  8,  64,  512  and  4096  block  cases.  Amongst 
the  files  in  that  directory  is  the  file  README-LPM3  which  gives  instructions  on 
how  to  use  LPM3.  A  copy  of  that  “readme”  file  is  attached  to  this  Report  in 
Annex  H. 

2.2  Subtask  1.2 

A  series  of  benchmarking  computations  have  been  performed  to  identify  the 
best  mode  of  operation  for  PIC3D  on  the  SP2.  Our  tests  show  that  the  native 
IBM  message  passing  library  gives  the  best  performance.  However,  the  speed 
gain  of  the  IBM  customised  version  of  PVM,  PVMe,  is  almost  as  good,  and  has 
the  advantage  of  maintaining  code  portability.  It  is  for  this  reason  that  we  have 
chosen  to  use  the  PVM  library  for  the  P1C3D  implementation  described  below. 

The  public  domain  version  of  PVM  3.0,  PVM3ip,  gives  considerably  poorer 
performance,  showing  slow-down  rather  than  speed-up  in  some  cases  as  the 
number  of  processors  is  increased.  The  fourth  option,  PVM3sw,  gave  better 
performance  than  PVM3ip,  but  inferior  to  PVMe  and  MPL. 

A  summary  of  the  SP2  benchmarking  results  and  a  comparison  with  figures 
on  the  Intel  Paragon  at  Sandia  are  given  in  Annex  A. 

2.3  Subtask  1.3 

LPM3  omits  much  of  the  physical  complexity  of  the  full  3DPIC  code  and  concen¬ 
trates  on  the  code  necessary  for  parallel  execution  and  performance  measure¬ 
ment.  In  this  subtask  the  main  simulation  code  PIC3D  was  advanced  to  the 
next  stage  of  its  development  cycle,  and  was  blended  with  the  parallel  code  from 
LPM3  to  give  the  version  of  PIC3D  which  operates  on  either  a  workstation  or 
parallel  computer.  A  number  of  refinements  have  been  made  to  the  simulation 
code  algorithm,  and  several  demonstration  calculations  have  been  undertaken 
on  both  the  SUN  Workstations  at  Culham  and  on  the  MHPCC  SP2  computer. 

Details  of  the  code  refinements  and  a  number  of  demonstration  calculations 
are  presented  in  Annexes  B. 

2.4  Subtask  1.4 

This  document  and  its  Annexes,  together  with  the  software  and  test  data  in¬ 
stalled  on  the  SUN  workstation  ppws20  at  Phillips  Laboratory  under  username 
eastwood  and  on  the  MHPCC  SP2  under  the  username  hockney  comprise  Sub¬ 
task  1.4  and  are  the  Final  Deliverables  of  this  contract. 

Installed  on  the  ppws20  workstation  in  directory  ~eastwood/bin-sun  are 
the  following  executables: 
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Executable 

Purpose 

peggie 

pic3d 

mpictim 

dispan 

tsdmerge 

input  preprocessor 
main  simulation  program 
time  series  data  postprocessor 
dispersion  analysis  postprocessor 
merge  .tsd  files  from  parallel  machine 

The  source  for  PIC3D  is  held  in  the  directory  3DPIC/PIC3D  and  support  libraries 
for  the  main  simulation  program  are  in  the  other  directories  under  3DPIC.  The 
directory  3DPIC/TEST  contains  the  .uif  files  for  the  example  cases  described 
in  the  Technical  Note  contained  in  Annex  B.  In  addition,  the  GHOST  graph¬ 
ics  library  was  previously  installed  for  displaying,  postprocessing  and  printing 
graphical  output.  A  number  of  test  datasets  have  been  provided  -  these  test 
cases  are  described  in  Annex  B.  The  preprocessor  peggie  and  postprocessors 
mpictim  and  dispan  were  not  developed  under  this  research  programme,  but 
are  supplied  in  executable  form  for  use  with  the  main  simulation  code  pic3d. 

Installed  on  the  SP2  are  the  source  and  parallel  executables  for  the  main 
simulation  program  PIC3D  and  the  benchmark  program  LPM3.  Also  pro¬ 
vided  on  the  SP2  is  the  additional  postprocessor  tsdmerge  for  merging  the 
timeseries  output  (*.tsd)  from  the  parallel  run  into  a  single  *.tsd  file  for 
transfer  to  the  workstation  for  postprocessing.  The  executable  tsdmerge  is 
held  in  directory  "hockney/bin  The  parallel  version  of  PIC3D  is  in  the  file 
"hockney/pvm3/bin/RS6K/pic3d.  The  source  for  the  PIC3D  code  is  in  the 
directory  MAUI-3DPIC.  For  further  details  see  Annexes  G  and  H. 
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Abstract 

The  impact  of  parallelisation  issues  on  the  3D  PIC  software  has  been 
studied  using  the  LPM3  benchmark.  Successful  studies  have  been  com¬ 
pleted  on  both  the  Intel  Paragon  (with  up  to  1840  processors)  and  the  IBM 
SP2  (with  up  128  processors).  The  best  option  for  maintaining  portability 
and  getting  speedup  on  the  SP2  is  the  per  processor  code  using  PVMe 
message  passing  routines. 


1  Background 

In  order  to  demonstrate  the  parallel  performance  of  the  new  3D  MIMD  PIC 
code  (3DPIC)  the  benchmark  version  of  the  code  (called  LPM3)  has  been  run 
on  the  Intel  Paragon  at  the  Sandia  National  Laboratory,  Albuquerque,  and  on 
the  IBM  SP2  at  the  Maui  High  Performance  Computer  Center  (MHPCC).  This 
extends  the  previously  reported  results  for  the  Paragon  and  the  Intel  iPSC/860. 

The  IBM  SP2  offers  several  different  communication  interfaces,  and  four  of 
these  are  compared:  Public  domain  PVM3  (using  ethernet  or  switch),  IBM  cus¬ 
tom  PVMe,  and  the  IBM  Message  Passing  Language  (MPL).  For  each  interface 
the  performance  of  both  the  per-patch  and  per-processor  versions  of  the  code 
have  been  measured. 

2  LPM3  Benchmark 

The  LPM3  benchmark  code  simulates  a  triply-periodic  three-dimensional  elec¬ 
tron  plasma.  The  plasma  space  is  divided  into  blocks,  each  of  which  contains 
512  particles  representing  the  electrons,  and  64  elements  on  which  the  fields  are 
calculated.  Prom  the  point  of  view  of  load  balancing  on  the  parallel  computer, 
the  block  is  the  smallest  unit  that  can  be  allocated  amongst  the  processors. 
The  problem  size  is  measured  by  number  of  blocks  Nb,  and  four  problem  sizes 
have  been  used  with  respectively  Nb  =  8,64,512,4096.  These  correspond  re¬ 
spectively  to  numbers  of  particles  Np  =  AK,  32K,  256K,  2M2  where  K  =  1024 
and  M2  =  K^.  The  timestep  is  such  that  about  ten  percent  of  the  particles 
leave  each  block  and  enter  neighbouring  blocks  during  a  timestep.  A  run  of  100 
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timesteps  is  chosen  as  the  benchmark  test  because  this  can  be  done  in  a  few 
minutes  for  problem  sizes  and  number  of  processors  of  interest  (tstep/s  in  the 
range  0.1  to  10),  and  the  conservation  of  total  number  of  particles  is  used  as  a 
validity  check. 

There  are  two  different  versions  of  the  benchmark  code,  which  are  selected 
by  the  value  of  the  last  input  variable  MXPASW.  The  per-patch  version  sends  a 
separate  message  for  every  patch  in  the  system,  and  there  are  9  patches  for  every 
block.  In  the  per-process  version,  on  the  other  hand,  the  patch  messages  are 
assembled  and  sorted  in  a  buffer  so  that  only  one  message  is  sent  to  every  other 
process  to  which  a  given  process  is  attached.  The  per-process  code  may  send 
10  or  100  times  fewer  messages  than  the  per-patch  version.  The  per-process 
version  should  be  significantly  faster  than  the  per-patch  version  on  computers 
with  a  high  message  startup  time  or  latency.  For  computers  with  low  latency 
there  will  be  little  difference  between  the  versions. 

3  IBM  SP2  Communication  Alternatives 

The  IBM  SP2  offers  the  following  communication  interfaces: 

PVMSip:  Public  domain  PVM  version  3.0  from  the  Oak  Ridge  National  labo¬ 
ratory  with  full  Internet  communication  protocol  (ip)  between  processors, 
using  ethernet  connections.  It  may  be  used  to  connect  processors  of  dif¬ 
ferent  types,  as  in  a  cluster  of  heterogeneous  workstations.  Word  format 
conversion  between  different  types  of  processor  is  accommodated.  For 
these  reasons  PVM3  has  a  high  startup  overhead,  but  it  is  implemented 
on  all  parallel  computers  of  importance.  PVM3  code  should  therefore  be 
highly  portable.  (Data  was  taken  at  MHPCC  on  a  dedicated  partition  of 
64  MByte  thin  nodes  on  4  January  1995). 

PVMSsw:  Same  as  above,  but  using  IBM  high-speed  switch  instead  of  ether- 
net  connections.  (Data  was  taken  at  MHPCC  on  a  dedicated  partition  of 
64  MByte  thin  nodes  on  4  January  1995). 

PVMe:  IBM  customised  version  of  PVM.  This  assumes  the  processors  are  all 
the  same  and  within  a  single  SP2.  Most  of  the  internet  protocol  can  be 
omitted  and  no  word  format  conversion  is  needed.  The  IBM  high-speed 
switch  is  used.  The  source  code  is  the  same  as  PVM3  and  therefore 
portable.  (Data  was  taken  at  MHPCC  on  a  dedicated  partition  of  64 
MByte  thin  nodes  on  7  March  1995,  using  PVMe  1.3.1,  xlf  3.2,  poe  1.2.1 
under  AIX  3). 

MPL:  This  is  the  ’’native”  IBM  SP2  Fortran  library  of  communication  subrou¬ 
tines.  This  provides  minimum  startup  overhead  in  a  high-level  language. 
However  MPL  is  unique  to  IBM,  and  MPL  code  is  therefore  not  portable 
to  other  computers.  The  high-speed  switch  is  used.  (Data  was  taken  at 
MHPCC  on  a  dedicated  partition  of  64  MByte  thin  nodes  on  7  March 
1995,  using  xlf  3.2,  poe  1.2.1  under  AIX  3). 
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In  order  to  compare  these  alternatives  we  show  in  Fig.  1  the  performance  of 
both  versions  of  the  LPM3  benchmark  on  the  8-block  case.  The  symbol  plotted 
identifies  the  version  used,  and  the  type  of  dotted  line  identifies  the  communica¬ 
tion  interface.  The  graph  plots  Temporal  performance  in  timestep  per  second 
(tstep/s)  against  the  number  of  processors  used  on  a  log/log  scale,  in  order 
to  show  the  scaling  of  performance  with  the  number  of  processors.  The  ideal 
linear  scaling,  in  which  the  performance  is  directly  proportional  to  the  number 
of  processors,  is  shown  by  the  dotted  line  at  45  degrees. 

Public  domain  PVM3  performs  badly  with  this  code,  with  the  per-patch  ver¬ 
sion  showing  no  speedup  over  single-processor  performance  with  either  PVM3ip 
or  PVM3sw  (lowest  pair  of  curves).  The  per-processor  version  shows  a  dis¬ 
tinct  improvement  (next  higher  pair  of  curves),  but  the  speedup  scarcely  im¬ 
proves  with  more  than  two  processors,  and  is  at  most  two  for  eight  processors 
(PVM3sw).  In  both  cases  use  of  the  switch  (sw)  is  slighly  better  than  ethernet 
(ip)  but  the  difference  is  not  significant.  The  advantage  of  combining  the  patch 
messages  into  a  single  longer  message  for  each  connected  processor  is  clearly 
seen. 

The  next  higher  pair  of  curves  are  for  IBM  PVMe  with  the  per-processor 
version  having  the  better  performance  for  all  processor  numbers.  The  step-like 
appearance  of  these  curves  is  due  to  load  imbalance  between  the  processors. 
For  1,  2,  4  and  8  processors,  each  processor  has  the  same  number  of  blocks, 
namely  8,  4,  2  and  1  block,  and  therefore  the  same  work  to  do.  The  load  is 
exactly  balanced  across  the  processors,  and  the  performance  is  optimal.  In 
other  cases  the  load  is  unbalanced  and  the  performance  is  determined  by  the 
processors  with  the  largest  number  of  blocks.  Other  processors  will  have  to  wait 
for  these  critical  processors  to  finish.  For  3  processors  the  largest  number  of 
blocks  per  processor  is  3,  which  explains  why  the  3  processor  performance  lies 
between  that  for  2  processors  (4  blocks  per  processor)  and  that  for  4-processors 
(2  blocks  per  processor).  For  5,  6,  and  7  processors,  there  is  always  at  least 
one  processor  with  two  blocks,  which  explains  why  the  performance  for  these 
three  cases  is  almost  the  same  as  that  for  4  processors  (2  blocks  per  processor). 
The  measured  performance  for  IBM  MPL  follows  the  pattern  as  for  PVMe  but 
is  in  most  cases  about  10  to  20  percent  better.  For  both  PVMe  and  MPL, 
there  is  little  difference  between  the  performance  of  the  per-patch  and  per- 
processor  versions,  showing  that  the  message  startup  time  is  sufficiently  small 
that  reducing  the  number  of  messages  sent  in  the  per-processor  version  produces 
little  saving  in  time. 


4  Comparison  of  Paragon  and  SP2 

In  order  to  compare  the  performance  of  LPM3  on  the  Intel  Paragon  with  the 
IBM  SP2,  we  show  in  Fig.  2  the  performance  of  the  per-processor  code,  which 
was  the  best  on  both  computers,  for  the  4  cases  of  8  ,64,  512  and  4096  blocks. 
The  Intel  Paragon  data  is  for  the  NX2  communication  library  running  under 
the  Sandia  SUNMOS  1.4.8  operating  system  which  may  be  considered  to  be 
the  Intel  equivalent  of  the  native  IBM  MPL  interface.  These  results  have  been 
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previously  reported.  Although  we  have  measured  both  versions  on  the  IBM  SP2 
under  both  MPL  and  PVMe,  we  plot  only  the  per-processor  results  which  were 
consistently  better  than  the  per-patch  results.  The  plotting  symbol  used  dis¬ 
tinguishes  the  problem  size  (number  of  blocks),  and  the  line  type  the  computer 
and  communication  combination. 

Looking  at  the  performance  for  one  processor  we  find  the  IBM  SP2  to  be 
about  2.5  times  faster  than  the  Intel  Paragon,  and  this  difference  in  single 
processor  performance  determines  the  other  results.  The  dotted  lines  at  45 
degrees  show  the  perfect  linear  scaling  from  the  one-processor  performance  for 
the  8-block  case.  This  assumes  that  performance  is  directly  proportional  to 
the  number  of  processors  and  inversely  proportional  to  the  number  of  blocks. 
The  Paragon  scales  extremely  well  right  up  to  1840  processors  which  is  the 
maximum  available,  and  this  enables  it  to  overcome  the  poorer  single-processor 
performance  in  the  512  and  4096  block  cases,  although  there  is  not  enough 
parallelism  in  the  64  and  8-block  cases  for  this  to  be  possible,  because  one 
cannot  use  more  processors  than  there  are  blocks.  The  SP2  performance  shows 
a  significant  fall  off  from  the  ideal  scaling  for  more  than  16  processors  on  the 
64-block  case  and  more  than  64  processors  for  the  512  block  case.  Although 
there  are  400  processors  on  the  Maui  SP2,  the  maximum  that  can  be  used  in 
practice  is  128,  so  it  has  not  been  possible  to  test  the  SP2  scaling  beyond  128 
processors. 

5  Conclusions 

For  the  best  performance  on  the  IBM  SP2  it  is  advisable  to  use  the  native 
communication  interface,  but  portable  PVM  code  run  under  IBM  customised 
PVMe  is  almost  as  good.  For  the  LPM3  benchmark,  public  domain  PVM3 
used  over  either  ethernet  (corresponding  to  a  cluster  of  workstations)  or  the 
switch  has  too  much  startup  overhead  to  give  any  worth-while  speedup.  The 
significantly  better  single-processor  performance  of  the  SP2  compared  to  the 
Paragon,  gives  the  SP2  an  initial  advantage,  but  this  is  lost  for  the  larger 
problems  by  the  better  scaling  of  the  Paragon  and  the  much  larger  number  of 
processor  that  are  available.  PVM  code  has  not  been  run  on  the  Paragon.  The 
per-processor  version  of  the  communication  interface  has  consistently  better 
performance  than  the  original  per-patch  version,  but  this  difference  is  only 
significant  for  PVM3  communications  when  it  is  dramatic. 
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Number  of  Processors 

Figure  1:  Temporal  Performance  of  the  LPM3  benchmark  measured  in  units  of 
timestep  per  second  for  the  8-block  case  on  the  IBM  SP2.  Four  different  commu¬ 
nication  alternatives  are  shown:  PVM3ip  (Public  Domain  PVM3  using  Internet 
protocol  over  ethernet  connections),  PVM3sw  (same  but  over  IBM  high-speed 
switch),  PVMe  (IBM  customised  PVM  over  switch)  and  MPL  (IBM’s  native 
Message  Passing  Language  over  switch).  Two  code  versions  are  shown:  (a) 
per-patch  and  (r)  per-processor.  Line  type  distinguishes  the  communication 
interface,  and  the  plotting  symbol  the  code  version. 
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Figure  2:  Temporal  Performance  of  the  LPM3  benchmark  (per-processor  ver¬ 
sion)  for  four  problem  sizes  on  the  Intel  Paragon  and  IBM  SP2.  The  plotting 
symbol  distinguishes  the  problem  size  (number  of  blocks)  and  the  line  type  the 
computer /communication  combination.  The  step- like  shape  of  the  Intel  results 
is  due  to  load  imbalance.  Except  for  the  8-block  case  the  IBM  results  only  show 
cases  of  exact  load  balance,  and  the  underlying  step-like  behaviour  is  not  seen. 
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Abstract 

This  Note  describes  work  to  enhance  the  particle  modelling  capabilities 
of  the  3D  PIC  suite  of  codes,  and  presents  a  number  of  sample  calculations. 
The  preprocessor  PEGGIE,  the  simulation  code  PIC3D,  output  capabilities 
and  the  postprocessors  MPICTIM  and  DISPAN  have  been  developed  to 
handle  particles  in  realistic  device  geometries,  and  thus  to  perform  and 
analyse  some  demonstration  electromagnetic  and  Particle-in-Cell  (PIC) 
calculations. 


1  Introduction 

Over  the  past  four  years,  AEA  Technology  at  Culham  Laboratory  has  been  de¬ 
veloping  a  new  software  suite  for  modelling  microwave  devices  [1].  The  software 
comprises  preprocessor,  simulation  and  postprocessor  modules.  The  preproces¬ 
sor  PEGGIE,  designed  to  make  the  setting  up  of  the  simulations  simpler  and 
less  error  prone,  provides  the  capability  to  prescribe  a  range  of  realistic  device 
geometry  configurations  using  a  ‘kit-of-parts’  approach  to  describing  the  geom¬ 
etry.  The  main  simulation  software  PIC3D  is  built  around  a  general  geometry 
field  solver  and  particle  dynamics.  The  postprocessors,  designed  to  display  and 
measure  snapshot  and  time  series  output  of  a  number  of  standard  line,  surface 
and  volume  quantities  are  at  present  limited  to  a  subset  of  electromagnetic 
quantities  and/or  specific  geometries.  These  software  modules  are  not  yet  fully 
mature,  but  are  sufficiently  well  advanced  to  show  that  they  will  provide  ef¬ 
fective  tools  for  computer  simulation  studies  of  not  only  parts  of  microwave 
devices,  but  also  (given  suitable  Massively  Parallel  Processor  Computers)  of 
entire  microwave  systems  with  hitherto  inaccessible  scale  lengths. 

Reported  here  are  the  results  of  a  four  month  research  and  development 
programme  to  progress  the  new  software  tools  to  the  next  stage  of  their  devel¬ 
opment  cycle  and  demonstrate  their  applicability  to  HPM  source  evaluation.  A 
vital  part  of  this  process  has  been  the  application  of  the  software  to  real  tube 
design  in  order  to  refine  the  physical  models,  algorithms  and  diagnostic  outputs 
of  the  software.  The  resulting  software  provides  powerful  new  capabilities  for 
electromagnetic  Particle-in-Cell  [2,  3]  modelling.  It  enables  three-dimensional 
(3-D)  modelling  of  microwave  tubes  and  microwave  transmission  where  the 
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interaction  of  electromagnetic  waves  with  charged  particle  flow  is  important 
[4,  5,  6],  and  can  equally  well  be  applied  to  other  time  dependent  problems 
normally  tackled  by  finite  difference,  time  domain  (FDTD)  codes. 

This  report  describes  work  designed  primarily  to  enhance  the  particle  mod¬ 
elling  and  output  capabilities  of  3DPIC.  The  next  section  (Section  2)  treats 
the  modifications  made  to  the  PEGGIE  preprocessor,  then  Section  3  explains 
the  changes  made  to  the  main  PIC3D  simulation  code.  Section  4  deals  with 
the  postprocessing,  and  Section  5  contains  sample  calculations.  Note  that  the 
latter  Section  also  demonstrates  a  model  (for  a  helix  Travelling  Wave  Tube  or 
TWT)  produced  without  using  PEGGIE. 

2  The  Preprocessor  PEGGIE 

The  long  term  goal  of  the  work  is  to  develop  an  easy  to  use  3-D  general  geometry 
PIC  code  for  simulating  the  evolution  of  electromagnetic  fields  interacting  with 
charged  particle  flows  in  the  complex  geometries  encountered  in  microwave 
sources.  An  essential  part  is  an  easy  to  use  preprocessor,  one  which  takes  a 
description  of  a  device  and  its  physics  that  is  comprehensible  to  the  trained 
engineer,  and  generates  the  data  needed  by  the  main  PIC3D  code. 

As  described  in  Ref  [7],  PEGGIE  is  an  interface  to  a  time  domain  solver  for 
Maxwell’s  equations.  Clearly  it  has  to  be  extended  to  initialise  PIC  calcula¬ 
tions.  Inspection  of  the  microwave  source  and  transmission  modelling  problem 
indicates  the  need  for  facilities  to: 

1.  impose  a  condition  on  a  surface  of  having  a  certain  impedance  together 
with  a  spatially  varying  electric  field,  to  be  referred  to  as  the  “surface 
condition” , 

2.  produce  line  plots  of  field  components  as  a  function  of  position, 

3.  plot  potential  differences  as  a  function  of  position, 

4.  simulate  the  emission  of  particles  from  surfaces, 

5.  produce  scatter  plots  of  particle  positions. 

The  incorporation  of  these  changes  into  PEGGIE  has  meant  the  updating  of 
Ref  [7]  to  produce  Ref  [8].  The  need  to  model  particles  leads  to  the  introduction 
of  a  tag  SP  to  define  a  species,  and  the  PA  tag,  which  defines  patches,  is  also 
modified,  so  that  patch  particle  physics  can  be  defined,  as  described  in  Ref  [8]. 

To  deal  with  line  plots  in  a  user-friendly  fashion,  it  is  possible  to  define  a 
plotting  domain  in  terms  of  a  curve,  and  the  curve  (CU  tag)  may  be  defined 
by  inheritance  and  extension.  In  other  words  we  can  define  a  curve  by  saying 
where  it  runs  in  a  given  block,  then  demanding  that  it  be  extended  through 
some  user  defined  set  of  blocks  called  the  “universe” .  It  was  decided  that  curves 
should  be  calculated  in  terms  of  the  co-ordinates  wherein  each  block  is  a  unit 
cube.  This  has  the  advantage  of  being  independent  of  the  local  meshing,  and 
the  conversion  to  the  element  based  data  used  to  generate  the  3DPIC  diagnostics 
is  straightforward.  While  line  extension  in  principle  is  straightforward,  because 
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PEGGIE  allows  complex  block  topologies,  problems  may  arise  at  block  edges, 
due  to  the  ambiguity  about  which  adjacent  block  is  to  be  sampled.  Unless  these 
are  resolved,  loop-like  curves  may  not  close  properly. 

The  obvious  way  to  resolve  this  difficulty  is  to  try  to  duplicate  an  element 
based  model.  In  essence,  the  element  based  approach  removes  ambiguities  by 
mapping  points  onto  a  lattice  which  connects  unambiguously  to  adjacent  blocks. 
Our  approach  is  lattice-free,  but  the  idea  of  having  a  near-by  point  which  be¬ 
longs  to  a  definite  glue  patch  and  therefore  enables  unambiguous  connection,  is 
taken  up.  Such  a  near-by  point  is  called  a  “fuzz-point”  :  we  choose  to  generate 
fuzz-points  by  mapping  local  co-ordinates,  €  [0,1]  to  rji  =  2(^i  —  1/2)  and 
calculating  77^  by  reducing  the  absolute  value  of  rji  by  0.1%  for  each  i  on  the 
block  surface  to  give  =  (1  +  Vi  )/2-  Thus  provided  no  very  tiny  blocks  are 
defined,  if  the  original  sits  on  the  edge  of  a  glue  patch,  the  fuzz-point  will 
not.  Once  the  connectivity  is  established,  the  point  corresponding  to  in  the 
adjacent  block  is  calculated  as  exactly  as  real  fioating-point  arithmetic  allows. 
The  only  ambiguity  emerges  when  is  at  the  centre  of  a  block  face,  ie  (0,0) 
in  the  Vi  system.  This  is  resolved  by  using  branch  directions,  which  essentially 
record  the  sign  of  (vi  —  Vi  )  for  fh®  previous  block  boundary  crossing,  and  use 
this  to  determine  the  new  Vi- 

Facility  (2)  for  plots  of  potential  against  position  means  essentially  plots  of 
J  E.dl  against  a  curve  normal  to  dl,  Here,  the  user  is  invited  to  define  secondary 
sub-domains  (explained  in  Ref  [9])  for  himself  using  the  SD  tag.  It  is  helpful  to 
note  that  these  sub-domains  must  also  be  two-dimensional  and  in  general  should 
be  full  joins  with  the  primary  domains,  as  this  is  assumed  in  the  diagnostic 
collection  routines  of  PIC3D. 

Facility  (5)  to  trigger  scatter  plot  production,  is  enabled  by  amending  the  FL 
tag,  to  allow  a  sampling  parameter  to  be  set.  This  prevents  excessive  numbers 
of  particles  appearing. 

The  checking  of  the  new  facilities  is  performed  using  the  input  datasets  or 
.uif  listed  in  Appendix  A.  Testing  is  now  a  two  stage  process,  since  PEGGIE 
is  no  longer  simply  generating  geometrical  information  which  can  be  inspected 
graphically,  but  complete  datasets  (or  .dat  files)  for  input  to  PIC3D.  Thus  the 
•  dat  files  have  first  to  be  generated,  then  PIC3D  run  to  validate  the  original 
.uif  files.  Hence  the  output  of  the  test  calculations  as  described  in  Section  5 
provides  the  validation  for  the  changes  to  PEGGIE. 


3  The  Main  Simulation  Program  PIC3D 

The  first  three  subsections  of  this  Chapter  describe  changes  made  to  the  main 
simulation  program.  The  final  subsection  contains  analytical  results  that  yield 
an  improved  initial  field  condition. 

3.1  Surface  Conditions 

To  treat  surface  conditions  requires  some  further  notation,  in  addition  to  that 
used  to  treat  resistive  walls  in  Ref  [10].  Equation  (3.25)  of  Ref  [10]  becomes 
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Table  1:  Entries  in  LBLAS 


Position 

Quantity 

Related  Field  (s) 

1 

INCP(l) 

^k )  )  7fc 

2 

INCP(2) 

(^ki  Pk )  7fc 

3 

ISPACE 

(^k »  0k )  7fc 

4 

lORTHO 

5 

INCP(l) 

dext 

6 

INCP(2) 

dext 

7 

ISPACE 

dext 

8 

null 

9 

INCP(l) 

7ii 

10 

INCP(2) 

7ii 

11 

ISPACE 

7ii 

12 

null 

13 

INCP(l) 

l2i 

14 

INCP(2) 

72i 

15 

ISPACE 

l2i 

e3  X  (ZI,  -  [E  -  Ee:.t])  =  0  (1) 

whence  it  follows  that 

h^  =  -{E^-EeJ)  (2) 

_  dext  )  ^2^ 

eo^/^ 

Consequently,  the  equivalent  of  (4.3)  is 
/ — 33^ 

I'  ^  -  ^[d^  -  d,J]),  k=l,2  (4) 

We  then  introduce  Xfc  =  2/(Z  +  20),  where  Z  =  eoZ/{Aty/g^^),  so  that  the 
update  Eq  (4.5)  of  Ref  [10]  remains  invariant  provided  now  (if  dext^  =  0  is 
assumed  to  avoid  possible  numerical  instability) 

7*  =  Ikidi  +  (5) 


Note  that  Xk  depends  on  k  because  and  are  defined  at  different  locations 
and  Z  may  depend  on  position.  Hence  it  is  necessary  to  define  how  scalar  data 
are  to  be  stored  on  surface  patches,  since  in  particular  this  permits  a  compact 
storage  for  the  case  when  Z  is  position  independent. 

The  three-dimensional  addressing  idea  used  in  Ref  [12]  is  modified  to 

LOG  =  1  -b  /  X  INCP(l)  +  J  X  INCP(2) 


(6) 
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Table  2:  Entries  in  BCATR 


Position 

Entry 

1 

2 

2 

NSIG 

3 

ISURF 

4 

dext 

Nd  +  4: 

//  2 

Uext 

2Nd  +  4 

2 

2Nd  +  5 

ai 

Nab  +  2Nd  +  5 

02 

2Nab  +  2Nd  +  b 

Pi 

^Nab  +  2Nd  +  5 

P2 

4Nab  +  2Nd  +  5 

Xl 

5Nab  +  2Nd  +  5 

X2 

^Nab  +  2Nd  +  5 

711 

Ny  +  QNab  +  2Nd  -h  5 

712 

2N-y  +  QNab  +  2Nd  +  5 

721 

3Nj  -f  6Nab  +  2Nd  -k  5 

722 

where  INCP  is  an  array  of  increments,  entries  of  which  will  be  zero  if  there  is 
no  dependence  on  the  corresponding  patch  co-ordinate.  LOC=l  corresponds 
to  the  ‘O’  point  of  a  block  and  the  patch  co-ordinates  are  directed  from  the 
‘O’  point  towards  the  ‘X’  point.  The  preferred  way  of  ordering  the  data  is 
to  have  the  patch  1  co-ordinate  information  stored  contiguously.  However, 
difficulties  were  encountered  in  routine  BCOPAT  of  PIC3D,  and  a  strategy 
allowing  INCP(l)  >  INCP(2)  was  adopted  as  an  interim  measure.  Subroutine 
MASKSA  was  written  to  test  the  dependence  of  scalar  field  s  on  position, 
SQASHS  to  perform  any  resulting  compressions  and  SCADR  to  set  up  INCP 
and  the  rest  of  the  patch  addressing  structure.  The  addressing  structure  for 
patch  IPATCH  starts  at  entry  LOBLAS  (NBTYPE  -I-  IPATCH)  of  the  array 
LBLAS.  The  entries  in  relative  position  are  given  in  Table  1. 

Since  they  vary  only  with  Z,  the  quantities  ak,/3k  and  Xfc  have  a  common 
spatial  dependence.  Note  that  ISPACE  is  the  total  space  occupied  by  a  field 
and  lORTHO  is  the  orthogonality  key,  denoting  whether  the  yki  are  present. 
The  coefficients  are  stored  in  array  BCATR  starting  at  location  MEMBCA 
(IPATCH)  -f-  1,  in  the  order  shown  in  Table  2.  NSIG  is  a  number  defining 
the  time  dependence  of  dext',  Nd,  Nab  and  N-y  indicate  total  scalar  storage  in  the 
obvious  way,  and  if  ISURF  is  non-zero  dext  is  defined  using  the  initial  vector 
potential  A,  as  will  now  be  explained. 

The  idea  is  to  initialise  b  globally,  using  existing  subroutines  and  variables, 
calculate  H  and  finally  V  x  H,  then  use  that  to  initialise  d.  Suppose  that  it 
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is  critical  to  have  a  certain  amplitude  of  E  or  potential  difference.  If  Zm  is  the 
mode  impedance  so  E  =  is  it  follows  (in  vacuo)  that 

E  =  ZjnB/fj,o  (7) 

Making  E,  Z  and  B  dimensionless  gives 

E  =  ZB  (8) 


where  here  and  in  the  remainder  of  this  subsection,  we  use  dimensionless  units. 

Thus  if  Eq,  initial  (dimensionless)  E  is  known,  the  initial  dimensionless  B 
needs  to  be  set  to  Eq/Z.  The  second  complication  arises  from  the  assumption 
that  E  =  Eosinwi,  where  cj  is  the  mode  frequency,  so  E  vanishes  at  t  =  0. 
However,  considering  the  discrete  relation 


E„+i  -  E„  =  V  X  H 


(9) 


implies  (t„  =  [n  —  1/2])  that 


2sin(a;/2)Eo  =  V  x  H 


Hence 


En  = 


-V  X  H 


(10) 


(11) 


2sin(a;/2) 

The  third  complication  is  caused  by  the  fact  that  V  x  H  is  not  fully  assembled 
at  a  boundciry.  This  is  corrected  for  by  doubling,  since  V  x  H  may  be  assumed 
to  be  symmetric  about  the  boundary  (if  it  is  antisymmetric  then  d  =  0). 
There  is  also  the  identity  for  z  =  0  that 


sin(.jt  =  ^(sin[a;t  +  kz]  +  sin[a;t  —  kz]) 


(12) 


which  is  another  way  of  saying  that  dgxt  generates  a  forward  and  backward 
wave.  Thus  the  total  factor  to  apply  to  do  is 

(13) 


Z  sin(a;/2) 

where,  in  terms  of  the  dimensional  frequency,  oi  =  =  27r/At.  This  fre¬ 

quency  factor  is  computed  in  PIC3D. 

More  complications  arise  because  the  amplitude  of  the  initial  6*  field  (not  B 
itself)  is  the  quantity  directly  specified  via  input  quantities  CMODE  or  BUNI. 
To  correct  for  this,  routine  PHYVEC  was  added  to  compute  B  from  the  initial 
b\ 

PHYVEC  is  essentially  a  diagnostic  routine,  and  PIC3D  operates  the  con¬ 
vention  that  all  diagnostics  are  element  based.  Even  so,  the  drawback  of  using 
the  relation 

B  =  (14) 

is  the  need  to  compute  y/g.  We  work  instead  with  the  formula 


B  ■  ej  X  efc  =  E 


(15) 
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where  (ijk)  is  a  cyclic  permutation  of  (123),  which  is  treated  as  a  set  of  three 
simultaneous  linear  equations  for  B,  given  6*.  In  the  element  based  approach, 
all  quantities  in  Eq  (15)  must  be  regarded  as  though  located  at  the  element 
centre,  so  the  ei  and  ¥  have  to  be  appropriately  averaged.  The  equation  solving 
approach  has  the  advantage  that  it  extends  easily  to  the  case  where  covariant 
components  (Hi)  are  given,  but  is  rather  expensive  unless  the  symmetries  of  the 
ei  are  exploited  (which  is  not  presently  the  case).  The  final  step  of  PHYVEC 
is  to  rotate  the  magnetic  field  components  into  the  global  coordinate  system. 

Thereafter  the  maximum  B  is  compared  with  the  expected  value  and  the 
ratio  CMODE/BUNI  is  adjusted  accordingly.  Once  the  initial  B  has  an  ampli¬ 
tude  given  by  this  new  CMODE/BUNI,  Eq  can  be  scaled  correctly  by  division 
by  CMODE  or  BUNI  in  the  PEGGIE  front-end. 

A  further  complication  is  that  the  boundary  may  be  at  a  node,  where  (T  =  0, 
of  the  initial  field.  To  prevent  this,  the  option  to  make  a  phase  shift  of  an  angle 
RPHSHF(j)  to  the  initial  conditions,  was  added.  Entry  j  =  1,2,3  corresponds 
to  {r,0,z)  or  {x,y,z)  in  the  global  co-ordinate  system.  Thus  the  fields  eg. 
computed  by  subroutine  TMREC  have  factors  of  the  form 

sin(rc  -b  RPHSHF(l))...  cos(.2  +  RPHSHF(3))  (16) 


3.2  Particles 

3.2.1  Particle  Equations 

Particle  momenta  and  positions  are  stored  in  terms  of  their  curvilinear  coor¬ 
dinates  local  to  the  block  which  contains  them.  The  momentum  coordinates 
are  updated  in  two  stages.  First,  the  momentum  components  are  updated  at 
the  old  particle  position,  then  are  transformed  to  components  measured  at  the 
new  position;  this  eliminates  the  explicit  appearance  of  the  Christoffel  symbol 
terms. 

The  established  centred  time  approximation  [2,  3,  13]  is  used  for  the  mo¬ 
mentum  update  at  the  old  particle  position.  Covariant  momentum  components 
k  at  time  level  n -1-1/2,  measured  at  the  current  particle  positions 

are  found  using 

(4"^  +  -b  ^Kn+l/2))tm(„)/2)  (17) 

where 

v''  =  p^/'ymo  (18) 

are  contravariant  velocity  components,  ekim  is  the  permutation  symbol,  and 
the  suffix  “p”  attached  to  At  reminds  us  that  the  particles  may  be  followed 
using  a  timestep  larger  than  that  used  for  the  field  advance.  Equation  (17)  is 
solved  using  the  method  due  to  Boris  [13];  apply  a  half  electric  acceleration, 
compute  the  relativistic  7,  apply  a  two  stage  rotate  and  then  complete  the 
electric  acceleration.  Fields  are  evaluated  at  the  particle  positions  using 

¥  = 


(19) 
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Ei  =  (20) 

where  the  basis  functions  and  were  defined  in  Ref.  [12]. 

Positions  are  updated  to  new  time  level  n  +  1  and  the  momentum  compo¬ 
nents  at  the  new  positions  are  computed  using  a  predictor-corrector  scheme  to 
solve 

^kin+l)  _  ^fc(n)  ^  J'-fc(n+l/2)(^(n))  ^  -fc(n+l/2)(-(n+l))  j  (21) 

and 

p'(«+i/2)(^(n+i))  ^  e^(s(”+i)) .  efc(s(”))p''("+^/^)(x("))  (22) 

In  cartesians,  this  scheme  reduces  to  the  usual  Lorentz  force  leapfrog  integration 
scheme  and  no  iteration  is  required.  In  cylindrical  coordinates,  the  predictor- 
corrector  can  be  replaced  by  direct  matrix  inversion  in  the  manner  proposed 
by  Boris  [13].  The  treatment  of  general  nonorthogonal  geometry  follows  in 
Section  3.2.2. 

In  addition,  we  have  the  current  assignment  relation,  that  computes  the 
currents  resulting  from  particle  motion  and  which  from  Ref  [12]  is  given  by 

f  dtJ2qpS^p^W^i){xl,xl,xl,t)  (23) 

''  p 

where  p  denotes  particle  index,  is  the  charge  on  particle  p,  ^  a  measure  of 
its  velocity  and  W^i)  is  a  weight  function  described  in  detail  in  Ref.  [12]. 
Schematically,  the  particle  equations  to  be  solved  are  for  kinematics, 

Ax^  =  v^Atp  (24) 

half  electric-field  accelerate, 

A/iPi  =  ^qEiAtp  (25) 

and  magnetic  field  rotate, 

ArotPi  ~  qAtpCijkP^ b  /q  (26) 

together  with  the  current  contribution  from  a  single  (super-)particle, 

=  qv^AtW  (27) 

where  IT  is  a  dimensionless  integral  of  the  weight  function. 

The  electromagnetic  fields  in  the  main  simulation  code  have  been  made 
dimensionless,  as  explained  in  Ref  [14],  therefore  it  is  necessary  to  convert  the 
new  quantities  appearing  in  Eqs  (24)  to  (27).  Let  us  introduce  the  subcycling 
parameter  Ns  so  that  Atp  =  NgAt,  and  adopt  the  convention  that  in  the 
remainder  of  this  subsection,  dimensional  quantities  appear  only  in  brackets 
(“[  ...  ]”).  If  the  dimensionless  version  of  Eq  (24)  is  taken  to  be 


Ax‘‘  =  v^N 


(28) 
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then  must  be  in  units  of  1/At,  whence  it  follows  that  p*  has  dimensions  mo/ At 
and  Pi  units  of  L^mo/At,  where  L  is  the  lengthscale  introduced  in  Ref.  [14], 
Equation  (25)  reduces  to 

AhPi  =  msEi  (29) 


provided 


ms 


qEoAtAtp  _ 
2moL  ^  ~ 


me 

mo 


)(i|)W 


and  similarly  Eq  (26)  becomes 


(30) 


ArotPi  =  ^mECijkpPh’^  h 


(31) 


In  these  units,  Eq  (27)  becomes 


V  =  4rAx^ 

Ns 

where  q  is  now  the  dimensionless  charge  on  a  species  superparticle 

_  r  9  1 

^  ^eoEoL2] 


(32) 


(33) 


As  explained  in  Ref  [8],  the  user  defines  each  species  in  terms  of  the  mass 
[m^]  and  charge  [g*]  on  a  single  particle,  together  with  the  number  Npg  of  such 
particles  to  be  represented  by  a  superparticle.  Thus  for  a  particular  species 
[mo]  =  A^ps[ms],  [5]  =  Aps[?s])  and  so  q/mo  =  qs/ms-  Species  1  consists, 
by  convention,  of  electrons.  The  simulation  code  PIC3D  works  in  terms  of 
the  particle  definitions  for  NSPEC  different  species,  that  appear  in  the  array 
SPATR,  as  shown  by  Table  3.  MPAINC  entries  are  allocated  for  each  species. 


Table  3:  Entries  in  SPATR 


Position 

Quantity 

Comments 

MONEPS  =  1 

Nps 

number  of  particles  per  superparticle 

MOCPS  =  2 

Npsqs 

dimensionless  charge  q  per  superparticle 

(may  be  negative) 

MRATPS  =  3 

[{me/ms){qs/\e\)]Ns 

m£;  in  particle  dynamics 

3.2.2  Particle  Dynamics  in  General  Geometry 

It  is  important  to  note  that  Eq  (26)  is  only  schematic.  The  Boris  scheme  uses 
a  time  discretisation  of  the  magnetic  field  acceleration  equation  of  form 


(34) 
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where  superfixes  ‘-’and  ‘-I-’  denote  values  before  and  after  respectively.  It  is 
easy  to  verify  that  Eq  (34)  is  exactly  soluble  in  terms  of 

(T  =  p“  -t-  p“  A  fi  (35) 


and 


as 


_  qBAt 
2mo7 


=  P  + 


2<t  A 
1  +  ^2 


(36) 

(37) 


Putting  Eqs  (35),  (36)  and  (37)  into  component  form  shows  that  gij,  and 
y/g  at  a  point  are  needed.  The  obvious  point  to  take  is  the  current  particle 
position,  and  it  is  natural  to  express  p  using  the  basis  at  ,  whilst  recalling 
that  p  is  leap-frogged  with  respect  to  x,  ie.  defined  at  a  time  t  =  {n+  ^)Atp. 

However,  it  is  not  sufficient  simply  to  store  components  of  the  metric  tensor, 
because  of  Eq  (22)  which  contains  a  product  of  basis  vectors  at  different  points. 
The  fundamental  geometrical  quantities  are  Cj,  e®  and  y/g,  whence  say  g^^  can 
be  obtained  as  ehe^  ,  etc.  The  vectors  are  already  stored  in  the  array  ECOV. 
Unfortunately,  y/g  and  particularly  e*,  are  not  so  easy  to  define  at  every  point 
from  an  array  as  their  variation  over  an  element  is  complicated.  The  approach 
adopted  is  therefore  to  compute  the  e*  and  y/g  at  x  from  the  ei(x).  It  follows 
that  it  is  necessary  to  limit  x*  such  that  <  ni,  because  eg  63  is  not  defined 
at  =  n^.  (There  is  no  problem  near  =  0  because  all  x*  G  (—1,1)  are  all 
mapped  to  zero  by  the  standard  FORTRAN  integer  truncation  routine.)  We 
do  not  consider  the  effects  of  round-off  error  any  further. 

The  next  and  most  important  issue  is  the  economical  evaluation  of  the 
geometrical  coefficients  needed  in  particle  pushing,  given  the  ej.  Prom  this 
point-of-view  the  two  main  properties  possessed  by  the  are  invariance  under 
coordinate  transformation  and  having  a  zero  entry.  The  former  makes  it  un¬ 
necessary  to  recompute  e^e^*  (the  denotes  a  basis  at  a  different  position)  in 
Eq  (22),  or  indeed  to  iterate;  the  latter  saves  multiplicative  operations  in  the 
dot  products.  However,  considering  first  the  invariance  property,  we  observe 
that  if  there  is  a  dependence  only  upon  one  co-ordinate  say  ei(5^),  this  trans¬ 
lates  into  rational  linear  dependences  on  5^  for  all  the  e*  (in  general)  so  making 
use  of  invariance,  even  in  this  simple  case,  is  not  easy.  The  only  case  perhaps 
worth  distinguishing  is  where  all  the  e,  are  invariant,  ie.  a  regular  gridding  of  a 
cuboid  or  parallelepiped.  However,  this  deserves  a  totally  separate  routine  for 
then  ehe^*  =  So  far  we  have  made  no  use  of  invariance  properties. 

The  most  obvious  simplification  due  to  null  e,  components  occurs  in  an 
orthogonal  block,  if  we  assume  the  local  coordinates  are  chosen  parallel  to 
the  Cj.  It  does  not  seem  unreasonable  to  assume  that  ei  ||  x,  62  ||  y  and 
63  II  z  where  x,  y  and  z  are  the  unit  vectors  of  the  local  coordinate  system. 
Clearly,  the  case  of  completely  general  ej  with  all  components  non-zero  must 
be  treated.  The  question  as  to  whether  any  other  types  of  are  worth  special 
treatment  may  be  answered  by  the  study  of  eij,  the  matrix  whose  columns  are 
the  components  of  with  respect  to  the  local  coordinate  system  (so  that  the 
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Cii  are  the  components  of  ei,  etc.)  The  inverse  of  Cij  has  the  sparsity  pattern 
of  the  corresponding  matrix  for  the  e®,  hence  the  only  worthwhile  intermediate 
case  emerges  as  the  (right)  cylinder  of  general  cross-section,  for  which  we  shall 
assume  the  sparsity  pattern 


e 


ij  — 


^  eii 
621 
V  0 


612  0  ^ 
622  0 
0  633  j 


(38) 


For  each  sparsity  pattern  we  shall  have  separate  routines  to  compute  e®, 
the  symmetric  inner  product  that  gives  5®^  or  5^,  the  general  inner  product  eg. 
for  e*.efc*,  and  the  matrix-vector  product,  eg.  gijp^. 

For  the  general  orthogonal  case  we  have 


6^  =  (1/611,0,0) 

=  (0,  1/622,0) 
6^  =  (0,0,1/633) 

\/9  =  611622633 


rriij  =  Xi-Yj 


For  the  right  cylinder 


where 


xiiyii  0 

0  3:222/22 

0  0 


ruijVj  = 


/  miiDi 

m22V2 

\  m33V3 


=  A  ^(622,-612,0) 
=  A“^(-62i,  611,0) 

e3  =  (0,0,1/633) 

y/g  =  633  A 

A  =  (611622  —  612621) 


(39) 

(40) 

(41) 

(42) 

(43) 


(44) 

(45) 

(46) 

(47) 

(48) 

(49) 


rriij  =  Xj.yi  = 


/  3:112/11  +  X212/21  3:112/12  +  3:212/22  0 

3^122/11  +  3:222/21  3:122/12  +  3:222/22  0 

0  0  3:332/33 


/  mii^i  +  mi2V2  \ 


m2lVi  +  m22V2 
m^3V3 


(50) 


(51) 


/ 


For  the  general  case,  there  are  no  simplifications.  The  e®  are  calculated 
using  : 
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where  {ijk)  are  a  permutation  of  (123),  and  all  operations  involve  full  matrices 
and  vectors.  Note  that  in  a  block  with  an  underlying  cylindrical  co-ordinate 
system  is  further  complicated  by  the  need  to  transform  to  a  cartesian 

frame  to  compute  the  dot  product. 

Once  the  geometrical  quantities  are  defined,  accelerating  the  particles  is 
straightforward,  given  the  replacement  of  cross  products  such  as  <t  A  fl  by 
y/geijkO^Q’^,  and  the  use  of  the  identity  ■  For  the  particle  move, 

a  number  of  schemes  are  possible,  once  it  is  realised  that  Eq  (21)  is  an  approx¬ 
imation  to 


j  v^dt  =  J  v.e^dt 

(53) 

V.  J  e^dt 

(54) 

for  a  velocity  v  which  is  independent  of  time.  The  choice  arises  because  e*  is 
typically  discontinuous  from  one  cell  to  another,  although  it  also  varies  within 
a  cell.  An  approach  that  involved  performing  the  e*  integral  as  a  distance- 
weighted  sum  over  the  initial  and  final  e*  was  tried  first.  However,  in  the  main, 
the  results  were  no  better  than  achieved  using  Eq  (22),  which  is  equivalent  to 
replacing  the  integral  (54)  by  the  average  of  e*  at  the  end  points.  Time  unfortu¬ 
nately  did  not  permit  the  investigation  of  other  quadratures;  the  discontinuous 
nature  of  the  e*  suggests  that  a  cell-by-cell  approach  might  be  the  best. 

Block  boundaries  are  already  treated  with  such  an  approach.  The  intersec¬ 
tion  of  the  trajectory  with  the  cell  boundary  is  calculated,  and  the  e*  at  Sj 
are  used  in  the  computation  of  the  e*  integral  for  the  corrector  step.  If,  at  the 
end  of  the  corrector  phase,  there  is  still  a  boundary  intersection,  p  is  expressed 
in  terms  of  the  basis  at  the  best  estimate  for  x\,  and  then  the  momentum  triple 
pr  (see  Section  3.2.3)  follows.  Thus  there  is  a  separate  predictor-corrector  loop 
for  each  block  traversed  by  a  particle  in  one  time-step. 

Lastly,  we  mention  the  approach  of  using  the  “straight  line  in  Cartesian 
geometry” ,  viz.  approximations  based  on  : 

Ax  =  j  vdt  (55) 

Suitable  formulae  for  the  iterative  solution  of  Eq  (55)  were  derived,  but  it 
appeared  that  their  implementation  would  be  more  expensive  than  the  “straight 
line  in  curved  space”  approach  described  earlier.  An  approximation  such  as 
replacing  each  cell  by  a  parallelepiped  might  be  competitive. 

3.2.3  Particle  Addressing 

As  explained  in  Ref  [12],  records  for  a  given  uniblock  IBLOCK  are  to  be 
found  in  LPARAS  and  COORDS,  starting  at  the  origins 
LOPARA(IBLOCK)  and  LOCOOR(IBLOCK)  respectively.  The  layout  of 
data  in  the  uniblock  record  in  COORDS  is  given  by  values  in  LPARAS.  The 
layout  of  LPARAS  is  summarised  in  Table  4.  Numerical  values  for  the  index 
locations  are  given  in  Table  5,  where  note  that  NOPARA  is  the  symbolic  value 
of  the  number  of  entries  in  an  addressing  record  in  LPARAS. 
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Table  4:  Contents  of  a  uniblock  record  in  LPARAS 


Location 

Entry 

MSPEC 

number  of  species 

MNPOS 

number  of  position  coords  stored 

MNMOM 

number  of  momentum  coords  stored 

MPNOO 

number  of  particles  of  species  of  species  1 

MPNOO  +  1 

number  of  particles  of  species  of  species  2 

MPNOO  +  MXSPEC  -  1 

number  of  particles  of  species  MXSPEC 

MPORO 

origin  in  coordinate  record  of  species  1 

MPORO  -k  1 

origin  in  coordinate  record  of  species  2 

MPORO  +  MXSPEC  -  1 

origin  in  coordinate  record  of  species  MXSPEC 

To  describe  the  contents  of  COORDS,  we  introduce 
INFOS  =  LPARAS(MNPOS),  INMOM  =  LPARAS(MNMOM)  and 
IPLEN  =  INPOS  +  INMOM.  Relative  to  the  origin  LOCOOR(IBLOCK), 
entries  1  ...  INPOS  of  COORDS  contain  the  contravariant  position  coordinates 
of  the  first  particle,  entries  INPOS  +  1  ...  IPLEN  the  covariant  momentum 
coordinates  of  the  first  particle,  entries  IPLEN  +  1  ...  IPLEN  +  INPOS  the 
position  coordinates  of  the  second  particle,  and  so  on  until  all  the  INSPEC 
=  LPARAS(MPNOO)  particles  of  the  first  species  have  been  treated.  The 
particles  of  the  second  species  follow,  starting  at  a  relative  location  given  by 
LPARAS(MPORO+l)  =  INSPEC  *  IPLEN  if  no  gaps  are  left. 

3.3  Particle  Boundary  Conditions 

The  patch  buffers  are  used  to  inject  particles  into  the  device,  and  to  transfer 
them  between  blocks.  First  we  describe  how  particles  may  be  injected,  then  the 
use  of  the  patch  buffers  is  explained. 

3.3.1  Particle  Emission 

The  particle  emission  algorithm  extends  the  scheme  used  for  two-dimensional 
problems.  The  main  difference  is  in  the  way  the  charge  density  Q  on  the  patch  is 
computed.  Subroutine  QSHONP  was  written  to  calculate  Q  in  a  hybrid  manner, 
viz.  using  nearest  neighbour  accumulation  in  the  directions  within  the  surface 
and  distance  (area)  weighting  in  the  normal  direction.  This  avoids  problems 
arising  at  patch  boundaries  where  Q  may  be  underrepresented  if  volume  (area) 
weighing  is  used  in  all  three  co-ordinates. 

In  subroutine  EMITEL,  for  each  row  of  elements  on  the  patch,  the  surface 
charge  in  each  element  is  computed  from  a  local  application  of  Gauss’  law. 
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Table  5:  Parameters  for  particle  addressing 


Name 

Value 

MSPEC 

1 

MNPOS 

2 

MNMOM 

3 

MPNOO 

4 

MPNOO  -f  1 

5 

MPNOO  +  MXSPEC  -  1 

11 

MPORO 

12 

MPORO  +  1 

13 

MPORO  +  MXSPEC  -  1 

19 

MXSPEC 

8 

NOPARA 

20 

which  gives 

a  =  d-Q  (56) 

where  d  is  the  normal  component  of  d}  (averaged  to  give  a  value  at  the  centre 
of  the  element  surface).  Two  passes  are  made,  the  second  interchanging  the 
row  and  element  indices  of  the  first.  On  each  pass,  particles  representing  cr/2 
of  charge  are  emitted  using  the  algorithm  devised  for  two-dimensional  emission 
on  each  row  or  column  (depending  on  the  direction  of  the  pass). 

The  emission  algorithm  thus  takes  the  following  form.  Let  x  —  where  i 
depends  on  the  direction  of  the  pass,  then  the  x-coordinate  of  the  next  particle 
emitted  is  found  by  summing  the  charge  a  over  the  line  (row  or  column)  of 
elements  in  x  until  it  first  amounts  to  more  than  one  electron.  A  seed  charge 
qo  where  0  <  g'o  <  1  is  used  to  start  the  process,  but  only  on  the  first  row 
of  the  first  pass.  Thus,  if  qjsf  is  the  charge  left  over  after  previous  creations  (so 
qN  =  Qo  a-f  the  start),  then  we  form 

Qt  =  QN  +  (^  (57) 

The  number  of  particles  emitted  per  element  of  the  pass  iem  is,  since  we  work 
in  units  where  the  electron  has  (negative)  unit  charge,  given  by 

iem  ~  *rit(  qy')  (58) 

where  int  denotes  integer  part.  When  iem  is  positive,  the  particles  have  x 
coordinate 

X  =  Xel  +  {i-  qN)/(^,  i  =  !,■■■, iem 


(59) 
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where  Xgi  is  the  lowest  value  of  x  on  the  element.  If  yei  is  defined  correspondingly 
for  the  patch  direction  normal  to  x,  then  the  particle’s  y  coordinate  is  set  to 
yei  plus  a  random  number  r  lying  between  zero  and  unity.  Its  initial  position 
normal  to  the  boundary  is  set  to  a  small  random  value.  Note  that  iem  is  not 
allowed  to  exceed  four  to  prevent  excessive  emission  during  transients, 

Particles  are  thought  of  as  having  been  born  at  a  time  (1  —  r)Atp  after 
the  start  of  the  present  particle  step.  Hence  they  initially  acquire  momentum 
through  being  accelerated  for  a  time  rAtp  in  the  local  electric  and  magnetic 
fields. 

Freshly  created  particles  are  introduced  into  the  main  calculation  by  placing 
them  in  the  particle  patch  buffers,  see  the  next  section.  They  then  appear  as 
though  they  have  just  crossed  from  a  neighbouring  block. 


3.3.2  Particle  Beam  Injection 

We  assume  that  the  beam  is  specified  via  its  current  profile  j(x),  and  the  energy 
of  the  beam  particles  (of  mass  mo  and  charge  q)  via  their  voltage  V.  The 
statement  that  a  beam  is  at  voltage  V,  then  translates  to  a  particle  energy  £ 
where  presumably 

£  =  moc^  +  qV  (60) 

and  mo(P  is  the  rest  mass.  However,  it  follows  from  special  relativity  that 

£  =  'ymQ(?  (61) 


where 

7  =  (1  -  ^)-^/^ 

(62) 

Squaring  Eq  (61)  and  using 

p  =  7mov 

(63) 

it  follows  that 

cl  2  4  1  22 

£  =  rriQC  -1-  p  c 

(64) 

and  hence  equating  to  the  square  of  Eq  (60) 

'2  f  V  ,  ^1 

P  —  22“(  2)"^^ 

mic^  \moC‘‘ J  ' 

r  qV  \ 

^,moc2/ 

(65) 

where  the  factors  have  been  arranged  so  that  each  of  the  three  terms  is  dimen¬ 
sionless. 

Eq  (65)  determines  the  momentum  of  each  particle  in  a  monoenergetic  beam. 
The  beam  current  density  determines  how  many  super-particles  need  to  be 
injected.  The  idea  is  that  particles  appear  on  a  surface  S  which  is  normal  to 
the  beam  direction  and  which  is  represented  as  the  union  of  a  collection  of 
surfaces  of  finite  elements.  Suppose  one  such  element  has  surface  vector  AS, 
then  the  current  passing  through  the  surface  is  j.AS. 

The  number  of  particles  to  be  introduced  per  time-step  in  an  element  is 
thus 


Nin  — 


j.ASAt 

1^1 


(66) 
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where  the  denominator  is  the  charge  on  a  superparticle.  (The  value  of  the 
right-hand  side  is  rounded  up  or  down  by  the  addition  of  r,  a  random  number 
in  the  unit  interval.) 

In  practice  j  is  normalised  using  the  charge  density  factor  po-  Observe  that 
for  a  monoenergetic  beam 

j(x)  =  vp(x)  =  povpp(x)  =  pojp(x)  (67) 


where  Pp(x)  and  jp(x)  are  profile  functions  for  the  charge  and  current  densities 
respectively.  Hence  if  the  total  current  is  I,  then 


1  = 


L 


yds  =  Po 


(68) 


where  po  is  defined  in  terms  of  beam  area  Ai,  and  particle  speed  v,  via 


/  =  povAb 


(69) 


The  speed  v  is  (from 
momentum  p  ,  by 


Hence 


Eqs  (63)  and  (65))  given  in  terms  of  the  dimensionless 


_  p'2c2 

1  +p^ 

(70) 

_  PojpAs 

NpsQs\^\ 

(71) 

where  As  is  the  area  of  the  element  surface  contributing  to  5.  Note  that  jp(x) 
has  to  be  normalised  so  that 


I  jp.dS  =  Abv  (72) 

Converting  to  dimensionless  units  gives 

Nin  =  Pojp-^s  (73) 


where 


Po 


•  I  1 

_Abv\  Npsqs\e\ 


The  dimensionless  momentum  follows  from  Eq  (65), 
defined  quantities,  dimensionless  voltage 


(74) 


where  in  terms  of  pre- 


'  qV  • 

.moc^. 


±.n}Ei[v] 

rUe  iVp5 


(75) 


where  niEs  is  entry  MRATPS  of  the  array  SPATR. 

Additional  refinements  allowed  by  the  software  include  uniform  beam  rota¬ 
tion  in  cylindrical  geometry  about  the  axis  (r  =  0),  and  it  is  anticipated  that  a 
finite  beam  temperature  will  be  allowed. 

One  difficulty  which  arises  in  practice  is  to  inject  a  beam  into  a  geometry 
where  the  edges  of  elements  do  not  conform  to  the  edge  of  a  beam,  eg.  the 
problem  of  injecting  a  circular  beam  in  a  cartesian  geometry.  This  can  be 
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Table  6:  Entries  in  BCATR  for  beam  injection. 


Position 

Entry 

1 

2 

2 

species  no. 

3 

NSIG 

4 

(temperature) 

5 

Mgeo 

6 

geometry  parameters 

Mgeo  6 

r 

Nj  -|-  Mgeo  +  6 

Pr 

Np  -|-  Nj  -f  Mgeo  +  6 

y^sN 

overcome  by  passing  the  coordinates  ^SN  (in  th®  global  system)  of  the  nodes 
in  the  particle  injection  surface  S,  together  with  Mgeo  parameters  describing 
the  beam  geometry.  In  the  case  of  a  circular  beam  geometry  Mgeo  =  2  to  store 
first  the  inner  beam  radius,  then  the  outer  one.  This  enables  PIC3D  to  reject 
particles  with  initial  positions  that  lie  outside  the  beam  cross-section. 

The  preprocessor  also  calculates  I',  the  number  of  particles  to  be  emitted 
per  element  in  each  At,  using  Gaussian  quadrature  to  evaluate  the  j  integral, 
and  the  momentum  triple  pr  corresponding  the  velocity  v.  The  coefficients  are 
stored  in  array  BCATR  starting  at  location  MPABCA  (IPATCH)  +  1,  in  the 
order  shown  in  Table  6.  NSIG  is  number  defining  the  time  dependence  of  Nj 
and  Np  have  the  obvious  meanings,  and  in  particular  for  a  three-dimensional 
geometry,  Np  =  3Nj. 

Fractional  I'  are  allowed  for  in  PIC3D  by  adding  r,  a  random  number  on 
the  unit  interval  to  /'(corrected  for  signal  strength  and  subcycling  effects), 
then  taking  the  integer  part  to  give  the  number  of  particles  to  be  emitted  on 
the  element.  Initial  positions  are  chosen  randomly  subject  to  the  geometrical 
constraints,  but  every  particle  in  the  same  element  is  assigned  the  same  pT- 

3.3.3  Patch  Buffer  Data 

Table  7  summarises  the  quantities  used  in  transferring  information  between 
uniblocks  via  the  gluepatch  buffers. 

To  transfer  from  block  1  to  block  2,  values  of  particle  exit  position  x\,  mo¬ 
mentum  triple  pt  (defined  below)  and  the  dimensionless  time  for  which  it  still 
has  to  move  are  copied  from  the  particle  arrays  in  records  of  length  MPB- 
POS  *  INPOS  -f  MPBMTM  *  INMOM  +  MPBATR  to  the  output  buffer 
GPATO.  Usually  MPBPOS  =  MPBMTM  =  MPBATR  =  1.  However,  in 
calculations  where  an  underlying  cylindrical  geometry  is  used,  MPBPOS  =  2, 
MPBMTM  =  1  and  MPBATR  =  0  consistent  with  the  patch  buffer  description 
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Table  7:  Gluepatch  exchange  buffer  data 


Location 

Value 

MBTOGP 

MGPTOB 

buffer  to  global  patch  table  pointers 
global  patch  to  buffer  pointers 

MPBEGI 

MPLENI 

NPINBI 

patch  start  address  in  input  buffer 
patch  lengths  in  input  buffer 
number  of  patches  in  input  buffer 

MPBEGO 

MPLENO 

NPINB 

NPADB 

patch  start  address  in  output  buffers 
patch  lengths  in  output  buffers 
number  of  patches  in  output  buffer 
pointer  to  first  free  location  in  GPATO 

LPHOC 

LPNXT 

particle  buffer  head  of  chains  array 
particle  buffer  link  list  array 

GPATI 

GPATO 

GPATT 

input  buffer 
output  buffer 
temporary  buffer 
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tion  in  [15].  At  the  same  time  as  the  copies  into  buffer,  information  on  the 
output  record  is  accumulated  in  MPBEGO,  MPLENO,  NPINB,  MBTOGP 
and  MGPTOB.  Patch  information  is  then  transferred  to  the  input  indexing 
arrays  MPBEGI,  MPLENI  and  the  input  data  buffer  GPATI,  and  thence  to 
the  target  block.  Any  permutations  and  sign  changes  needed  are  performed  on 
the  patch  buffer  data  in  subroutine  PCXFRM  which  is  called  by  PARTIO. 
The  pointer  tables  MBTOGP  and  MGPTOB  axe  used  to  access  global  infor¬ 
mation  on  block,  process,  etc.  Since  particles  can  exit  a  block  in  random  order 
through  any  face,  they  need  to  be  sorted  into  patches,  and  it  is  for  this  that 
the  linked  list  addressing  quantities  LPHOC  and  LPNXT  are  introduced.  In 
addition,  particles  can  cross  more  than  one  block  boundary  in  a  timestep,  and 
this  leads  to  the  need  for  an  extra  buffer  GPATT. 

The  need  to  introduce  a  momentum  triple  is  now  explained.  Transforma¬ 
tions  consisting  of  permutations  and  sign  changes  are  adequate  for  position 
values  (except  that  care  has  to  be  taken  so  that  x*  <  nj  after  the  positional 
transformation).  However,  in  general  there  is  a  more  complicated  transfor¬ 
mation  for  Pi,  because,  supposing  the  normal  coordinate  to  be  i  =  3  in  each 
block,  63  is  not  always  continuous  across  block  boundaries.  The  vector  is 
continuous,  suggesting  that  sign  changes  and  permutations  of  the  mixed  triple 
{pi,P2iP^)  might  be  adequate.  In  fact  the  momentum  has  to  be  represented  in 
buffer  as  pr  =  {pi,P2-,  \/9P^)  (or  appropriate  perturbations  depending  on  the 
normal  direction),  where  y/g  is  the  volume  element  of  the  block  which  is  left. 

To  see  this,  note  that  on  the  block  boundary,  continuity  implies 


p  =  Pie*  =  pje* 

(76) 

where  undashed  quantities  belong  to  the  block  (1)  which  is  left,  and  dashed 
ones  to  the  block  (2)  which  is  entered.  By  construction  ei  and  62  are  the  same 
for  each  block.  Since  for  (ijk)  permutations  of  (123),  by  definition 

i  ej  A  ek 

^  V9 

(77) 

and  it  follows  that  dotting  Eq  (76)  with  ei  and  62  respectively  gives 

Pi  =  Pi 

(78) 

P2=P2 

(79) 

Thus  the  tangential  momentum  is  continuous.  To  treat  the  normal  direction, 
note  that,  analogously  to  Eq  (78), 

p  =  pei=p  Si 

(80) 

Dotting  with  gives 

P^  = 

(81) 

and  upon  using  Eq  (77), 

V9P^  =  Vgp^' 

(82) 

Hence  the  inter-block  transfer  is  a  three-stage  process  : 
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(i)  Compute  pr  =  (p■el,p.e2,p.e^^/g)  upon  discovering  that  block  1  is  to 
be  left 

(ii)  Apply  permutation  and  sign  changes  (due  to  the  different  orientations  of 
two  blocks’  coordinate  systems)  to  pr  in  the  particle  patch  buffers 

(iii)  Compute  =  (pti,PT2,P3)  on  entry  to  block  2.  The  quantity  p^  is 
given  by  : 

,  Ptz-'/F^^'- 

Pi  '  0/  o/ 

y/g  S'*  .e^ 

which  follows  from  the  identity  p^  =  Ylj  9^^Pj 
Note  that  if  block  1  is  orthogonal 


PT3  =  ^3 


611622 

^33 


(84) 


where  6(q(j)  =  and  superscript  s  denotes  values  at  the  start  of  the  time-step. 
If  block  2  is  orthogonal,  Eq  (83)  simplifies  to 


P3 


633 

611622 


PT3 


No  other  worthwhile  simplifications  seem  possible. 


(85) 


3.4  Modifying  Initial  Conditions 

Let  us  introduce  the  magnetic  vector  potential  A(x,  t).  For  A  to  represent  an 
eigensolution  of  the  discrete  equations  solved  by  PIC3D,  it  must  satisfy 

-  dt^A  =  X  X  A  =  cojA  (86) 

where  dt  and  are  the  discrete  representations  of  the  time  differentiation  op¬ 
erator  and  the  gradient  operator  respectively,  and  G^  are  the  metric  tensors 
and  bJd  is  the  approximate  eigenfrequency.  The  initial  conditions  employed  by 
PIC3D  use  Aan,  the  solution  of  the  continuous  version  of  Eq  (86) 


VxVxA  =  up' A 

(87) 

The  two  potentials  will 

in  general  differ  slightly,  so 

A  =  Aan  +  6 

(88) 

Writing 

Lh  =  G^Vd  X  G^VdX 

(89) 

it  follows  that 

{Lh  )Aan  ~  {^d 

(90) 

thus  if 

LffC  — 

(91) 

then 

{Lh  up')  Aan 
{uP-\^) 

(92) 
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It  follows  that  the  field  A  should  be  a  more  accurate  solution  of  the  discrete 
Helmholtz  equation,  where 


(Lff  -  A2)  Aa„ 


(93) 


The  significance  of  the  Lff  operator  is  that  its  effect  is  equivalent  to  one  pass 
through  the  main  time  step  loop  of  PIC3D,  thus  the  amount  of  extra  coding 
needed  is  kept  to  a  minimum.  In  Eq  (93)  the  quantity  A  is  estimated  using 


2  _  (Lffe,Lffe} 

(e,e) 


(94) 


where  (, )  denotes  an  inner  product  on  the  space  of  vector  functions.  There 
are  a  number  of  ways  to  estimate  oJd,  of  which  0  is  employed  for  the 

magnetostatic  problem  considered  below.  Another  possibility  is 


2  {LjjA.am-^an) 
LO  — 


{■^ani  ^an) 


(95) 


In  practice,  the  inner  product  is  replaced  by  the  sum  of  an  element-wise, 
component-by-component  product  of  vectors. 


4  Postprocessors  MPICTIM  and  DISPAN 

There  are  now  two  main  codes  for  postprocessing  PIC3D  output,  MPICTIM 
which  was  described  in  Ref  [16]  and  DISPAN  for  dispersion  analysis.  The  en¬ 
hancements  to  MPICTIM  should  be  apparent  to  the  user,  and  easily  assimilated 
by  anyone  familiar  with  Ref  [16].  The  dispersion  analysis  postprocessor  DISPAN 
allows  the  interactive  interrogation  of  the  “.lin”  datasets  generated  by  PIC3D. 

The  dispersive  properties  of  a  microwave  structure  are  measured  by  setting 
up  a  periodic  simulation  containing  an  integral  number  of  cavities,  and  collect¬ 
ing  data  as  a  function  of  space  (axial  position)  and  time.  The  data  collected 
and  its  sampling  frequency  is  user  selected  in  the  same  manner  as  for  the  time 
series  data  sets  used  by  MPICTIM.  The  user  then  follows  the  DISPAN  operat¬ 
ing  sequence  to  extract  frequency  and  wavenumber  information  from  datasets 
generated  by  PIC3D. 

DISPAN  takes  the  user  through  the  following  sequence  of  operations: 

1.  file  selection 

2.  domain  selection 

3.  field  selection 

4.  spatial  fourier  transformation 

5.  selection  of  time  interval 


6.  time  FFT 
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7.  structure  factor  display 

8.  frequency  spectra  at  given  k 

9.  further  selection  options 

The  input  to  the  dispersion  analysis  program  DISPAN  is  a  two  dimensional 
array  of  numbers  A{p,q)  =  A{zp,tg)-,  p  =  l...NZ,q  =  1...NT  and  arrays  of 
position  coordinates  {zp  :  pe[l,NZ]},  and  time  coordinates  {tq  :  qe[l,NT]}.  In 
the  present  version  of  the  software,  it  is  assumed  that 

1.  Zp  =  Zo  +  {p  —  l)Az,  where  Az  is  the  spatial  mesh  spacing,  and  there  is 
period  length  L  =  NZ  x  Az,  i.e. 

A{z  +  nL,  t)  =  A{z)-,  n  integer. 


2.  NZ  =  2^3^5*  where  r,  s  and  t  are  integers  >  0. 

To  reduce  the  length  of  computations  required  to  construct  the  structure 
factor  S{f3,  /),  we  assume  that  A  is  periodic  in  time  rather  than  use  the  usual 
cosine-bell  envelope  over  many  cycles  of  oscillation.  Interactive  selection  of  the 
temporal  periodicity  allows  the  period  to  be  chosen  to  fit  the  period  of  one 
of  the  longer  wavelength  modes  (eg  the  mode  with  /?  =  ±27r/L);  this  allows 
reasonably  accurate  estimates  of  the  eigenfrequencies  /  =  /(/?)  of  the  long 
wavelength  modes  to  be  obtained  from  just  one  or  two  periods  of  oscillation. 

The  stages  of  analysis  are: 

1.  Spatial  fourier  transformation. 


A{k,q) 


NZ-l 

^  A{p,q)exp 

p=0 


i2'Kkp 

NZ 


(96) 


2.  The  period  selection  process  selects  a  subset  of  data  values  {tg,  A{k,q): 
qe[qL,qu]  }  taken  to  be  an  integral  number  of  oscillation  periods. 

3.  Cubic  spline  fitting  to  {tq,  A{k,q)  }. 

4.  Interpolation  to  a  uniform  net  of  NT  NEW  =  2^  modes,  where  2^“^  < 
qu  ~  Ql  +  1  <  2^  and  the  time  interval  between  nodes  DTNEW  = 
(iqu  ~  tg^)lNTNEW.  The  results  is  the  set  of  values  A^k,  q) 

5.  Temporal  fourier  transformation 


A{k,L) 


NTNEW-l  ^ 

Yj  A\k,q)exp 


q=0 


i2TTlq 


NTNEW 


(97) 
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6.  Conversion  to  amplitude-phase.  The  FFT  software  used  transforms  N  real 
data  values  to  iV/2  -t- 1  real  and  imaginary  part  values  for  its  hermitian 
transform.  The  doubly  transformed  data  A{k,  1)  is  stored  as  the  sum  of 
four  numbers  for  A:e[0,iVZ/2],  l€[0, NTNEW/2]: 

A{k,  e)  =  Aee  —  Ago  +  ii-Aeo  d-  Age)  (98) 


where  subscripts  ‘e’  and  ‘o’  denote  even  and  odd  in  the  indices  i.e. 
Aeo{k^l')  ~  Ago{,  0 

=  Agg^k,  1) 

=  -Aggi-k,-l)  (99) 

and  so  forth.  Amplitudes  S  and  phases  (p  for  A'Z/2  <  k  <  NZ/2,  0  < 
/  <  NTNEW/2  are  given  respectively  by 

S  =1  i  h  \l{Agg-Aggf  +  {Agg  +  AggY  (lOO) 


(f>  =  arctan 


/  Ago  A  Agg 
V-d-ee  Agg 


(101) 


Amplitude  5(/?, /)  =  S{k,l),  where  frequency  /  and  wavenumber  P  are 
related  to  I  and  k  by 


(jJ  _  I  „  _  2nk 

^  ^  NT  NEW  At'  ^  ~  NZAz 


(102) 


The  bidirectional  spectrum  gives  contours  of  S  in  {k,  f)  space,  or  sections 
of  S  versus  /  for  given  values  of  k.  The  ratio  of  5(— /3,  /)  and  5(/?,  /)  give  the 
ratio  of  amplitudes  of  forward  and  backward  waves;  if  S{—P,f)  =  0,  then  the 
wave  is  purely  left  going  ~  exp[i(u;t  -I-  Pz)]  and  if  S{P,  f)  =  0,  it  is  purely  right 
going  ~  exp[i{ujt  —  Pz)].  For  an  example  of  a  plot  of  5,  see  Fig  6  in  Section  5. 

The  omnidirectional  spectrum  gives  the  amplitude  as  a  function  of  P  and  / 
irrespective  of  direction: 


S'  —  \J A\g  A  Alg  +  Alg  -I- 


(103) 


5  Sample  Calculations 

This  Section  is  inevitably  something  of  a  picture  show.  Each  of  the  test  prob¬ 
lems  defined  by  the  PEGGIE  “.uif”s  in  Appendix  A  is  described,  and  a  sample  of 
the  consequent  output  from  PIC3D,  visualised  using  MPICTIM  unless  otherwise 
stated,  is  presented.  Finally  the  simulation  of  a  helix  TWT  is  described. 
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5.1  Connectivity  Test  une51 

To  test  the  treatment  of  inter-block  connectivities  both  in  PEGGIE  and  PIC3D, 
a  grid  was  constructed  using  eight  one  metre  cubes  that  are  joined  (full-face) 
in  every  possible  way.  The  result  of  attempting  to  make  every  block  have  a 
mesh  3  X  4  X  5  is  drawn  in  Fig  1.  Note  that  the  geometry  is  periodic, 
thus  which  of  the  images  of  a  block  appeals  depends  on  arbitrary  details  of  the 
PEGGIE  connectivity  algorithm.  However  PEGGIE  has  successfully  produced  a 
consistent  meshing,  and  the  corresponding  dataset  une51.dat  ready  for  input 
to  PIC3D. 


Figure  1:  The  meshing  produced  for  the  connectivity  test  une51.  Each  block 
is  a  metre  cube. 

The  input  dataset  is  designed  to  excite  a  mode  with  n  =  m  =  p  =  1 

of  wavelength  2  metres  in  each  coordinate  direction.  Such  a  mode,  in  a  cavity 
of  dimensions  a  x  b  x  d,  oscillates  with  a  frequency  of 


where  c  is  the  speed  of  light.  Hence  at  any  given  station  for  uneBl  parameters, 
a  regular  oscillation  at  /  =  259.8  MHz,  corresponding  to  a  period  of  3.85ns, 
is  to  be  expected.  Figure  2,  a  run  of  100  steps  with  timestep  At  =  3.849  x 
10“^®s,  bears  out  this  prediction.  The  amplitude  of  the  electric  field  reflects  a 
dimensionless  value  of  Eq  =  0{\),  since  this  translates  to  a  dimensional  value 
of  Eq  =  2mec/(|e|At)  w  1.8  x  10®//i  for  unit  Courant  number,  where  h  is  a 
measure  of  mesh  spacing  in  metres. 
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Figure  2:  Point  values  of  electric  field  component  in  Pm  ^  plotted  as  a 
function  of  time  t  in  fis  for  the  PIC3D  run  une51. 

5.2  Nonorthogonal  Test  tp06 

To  test  electromagnetic  phenomena  in  a  nonorthogonal  geometry,  a  grid  was 
constructed,  consisting  of  a  single  parallelepiped  with  periodic  boundary  con¬ 
ditions  in  each  direction.  The  parallelepiped  took  the  form  of  a  cylinder  of 
height  2  m,  which  in  cross-section  is  a  parallelogram  of  sides  2  and  2\/2  metres 
and  angle  of  45°.  After  some  minor  changes,  PEGGIE  successfully  produced  a 
dataset  containing  the  correct  patch  information. 

The  initial  condition  was  a  TEm  mode  in  the  unit  cube,  which  provides  a 
spatially  triply  periodic  solution  to  Maxwell’s  equation  in  the  cube  of  side  2. 
The  idea  behind  the  choice  of  geometry  of  the  parallelepiped  is  to  provide  an 
alternate  “tiling”  of  three-space.  Thus  this  meshing  is  simply  a  strange  way  of 
representing  the  unit  cell  of  a  triply  periodic  geometry. 

Figure  3  shows  that  at  the  station  in  the  centre  of  the  parallelepiped,  the 
electric  field  exhibits  beats.  The  run  involved  200  timesteps,  the  mesh  used  was 
10  X  10  X  10  and  the  timestep  At  =  2.722  x  10“^*^s. 

Beats  are  to  be  expected  on  the  basis  of  the  following  argument.  The  TEm 
mode  in  cartesian  coordinates  corresponds  to  a  magnetic  vector  potential 

A  =  (cos  Ttx  sin  ny  sin  Ttz,  —  sin  irx  cos  Try  sin  tt^,  0)  (105) 

The  nonorthogonal  geometry  defined  by  tp06  has  basis  vectors  given  by 

ei  =  (6, 0, 0),  62  =  (6,  b, 0),  63  =  (0, 0,  b)  (106) 

where  6  is  a  constant,  the  exact  of  value  of  which  need  not  concern  us.  Hence 
the  components  of  A  in  this  system  are  Aj  =  A.ej,  or 


Ai  —  bAxi  A2  —  b{Ax  +  Ay),  A3  —  bA 


(107) 
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implying  in  the  appropriately  normalised  nonorthogonal  system  that 

Ai  oc  cos(a:^  +  x^)  sina:^  cos  a?^,  yl2  —  sinx^  sinx^,  yls  =  0  (108) 

Thus  the  Aj  are  a  mixture  of  different  spatial  dependences.  Analytically  these 
must  have  same  frequencies,  since  the  spatially  triply  periodic  extension  of  the 
TEiii  mode  has  a  unique  frequency,  however  eccentrically  the  unit  cube  is 
meshed.  However,  discretisation  effects  will  mean  that  the  different  spatial  de¬ 
pendences  correspond  to  slightly  different  propagation  frequencies,  that  is  to 
say,  beats.  To  check  further,  the  dispersion  relation  in  Ref  [11]  was  evaluated 
for  modes  with  wave-vectors  m  =  (1,0,1)  and  (1,2,1)  in  the  curvilinear  sys¬ 
tem,  i.e.  we  set  kj  =  rajir/Nj  where  Nj  is  the  number  of  elements  in  the  j 
direction.  The  predicted  frequencies  are  /i  =  269  MHz  and  /2  =  313  MHz 
respectively,  corresponding  to  a  mean  frequency  of  291  MHz  (period  3.44  ns) 
and  a  beat  frequency  of  22  MHz  (period  45  ns).  These  agree  satisfactorily  with 
the  computed  values  of  3.3  ns  and  50  ns  respectively. 
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Figure  3:  Point  values  of  electric  field  component  in  Pm  ^  plotted  as  a 
function  of  time  t  in  /xs  for  the  PIC3D  run  tp06. 


5.3  Oscillations  on  Axis  cyl07 

The  dataset  cyl07  was  devised  to  investigate  the  appearance  of  spurious  oscil¬ 
lations  localised  near  the  axis  of  a  cylindrical  device,  when  non-axisymmetric 
TElll  and  TMm  modes  are  used  as  initial  conditions.  To  reduce  the  problem 
to  essentials,  cyl07  involves  a  single  block  representing  a  cylindrical  cavity  in 
perfectly  conducting  material  which  is  1  mm  in  radius  and  2  mm  long.  The 
magnetic  field  is  initialised  as  a  constant  vector  pointing  in  the  transverse  x- 
direction:  in  the  absence  of  electric  field,  this  is  a  steady  state  solution  of 
Maxwell’s  equations. 

Figure  4  is  a  superposition  of  results  firom  two  runs,  without  and  with  the 
correction  to  the  initial  conditions  discussed  in  Section  3.  (The  amplitude  in  the 
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plots  is  essentially  arbitrary  since  the  6  and  r  components  of  H  become  multiply 
valued  at  the  origin.)  The  former  curve  possesses  the  spurious  oscillation,  the 
latter  shows  it  at  much  reduced  amplitude.  In  each  case  a  mesh  8x8x16  was 
used  with  a  timestep  At  —  9.447  x  10~^^s.  The  conclusion  to  be  drawn  is  that 
the  spurious  oscillations  are  simply  a  consequence  of  the  failure  of  the  initial, 
analytic  fields  to  solve  the  relevant  discrete  Helmholtz  problem  for  a  cylindrical 
cavity.  Hence  for  many  purposes  they  may  be  neglected. 
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Figure  4:  The  value  near  the  origin  of  the  magnetic  field  component  Hr  in 
arbitrary  units  plotted  as  a  function  of  time  t  in  ps  for  the  PIC3D  run  cyl07. 


5.4  Surface  Conditions  sbcl6z 

The  problem  was  to  test  the  surface  conditions.  A  TMm  mode  hcis  a  frequency 
of  25.98  GHz  in  a  cubical  cavity  of  side  10cm.  The  corresponding  relative 
impedance  is  given  by 


Zr 


1  + (2^,2  + A 

ap  bp 


(109) 


for  a  general  TMnmp  mode  in  a  cavity  of  dimensions  a  x  b  x  d.  Thus  setting 
n  =  m  =  p  =  l,  a  =  b  =  d  =  0.01,  gives  Zr  =  0.5769,  corresponding 
to  Zg  =  217.51fl.  A  TMii  mode  of  axial  wavelength  2d  excited  in  such  a 
geometry  with  the  corresponding  impedance  at  entry  and  exit  planes  should 
travel  as  though  in  an  infinite  waveguide. 

The  coefficients  in  the  file  sbcl6z.dat  generated  by  PEGGIE  were  checked 
by  hand,  then  the  dataset  was  used  to  control  a  PIC3D  run.  Fig  5  shows  that, 
after  an  initial  transient,  point  values  of  Ey  at  different  z,  co-ordinate  along  the 
waveguide,  fit  the  pattern  of  a  mode  propagating  along  an  infinite  waveguide. 
Details  of  the  computation  are  that  it  used  an  6  x  8  x  10  mesh  and  timestep 
At  =  1.925  X  10“^^s  corresponding  to  unit  Courant  number. 
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Figure  5:  Point  values  of  Ej,  at  different  2,  plotted  as  functions  of  time  in  ps, 
for  the  model  rectangular  waveguide  in  run  sbcl6z. 


5.5  MILO  Cavity  Dispersion  nilp903 


Here  the  geometry  is  one  segment  of  an  annular  Magnetically  Insulated  Line 
Oscillator  (MILO),  ie  two  concentric  annular  cavities,  the  outer  of  which  has 
smaller  axial  extent  than  the  inner.  Periodic  boundary  conditions  are  imposed 
in  the  axial  direction,  to  represent  a  device  of  infinite  length  for  which  analytic 
field  solutions  are  available.  The  dimensions  are  (outer  cavity  in  parenthesis), 
inner  radius  =  3.75cm  (7.5cm),  outer  radius  =  7.5cm  (15cm),  axial  extent  =  5cm 
(4cm).  Each  annular  cavity  is,  for  meshing  purposes,  divided  in  angle  into  three 
equal  segments,  each  of  which  has  a  8  x  4  x  20  (8  x  4  x  16)  mesh,  implying 
8  elements  in  the  radial  direction,  4  in  polar  angle  0  and  20  (16)  elements  in  2: 
along  the  axis. 

In  order  that  a  range  of  wavelengths  is  excited  large  enough  to  compare 
with  analytic  results,  the  dataset  mlp903.uif  represents  a  nine-cavity  MILO. 
The  ability  to  define  a  line  by  extension  and  inheritance  is  particularly  use¬ 
ful  here,  where  a  line  parallel  to  the  z-axis  crosses  nine  blocks:  the  resulting 
plots  were  most  helpful  in  checking  the  initialisation  procedure.  The  analytic 
comparison  involves  purely  electromagnetic  waves  in  vacuo,  but  the  method  of 
excitation  involves  switching  on,  for  the  first  timestep  only,  a  current  directed 
along  z  in  one  MILO  segment.  The  run  mlp903  had  At  =  4.750  x  10~^^s  and 
continued  for  5ns,  during  which  time  data  were  output  to  a  .lin  file.  Subse¬ 
quent  analysis  by  DISPAN  produced  the  contour  plot  S(k,l)  drawn  in  Fig  6. 
Overlaying  the  contours  are  dispersion  curves  obtained  analytically,  assuming 
periodicity  and  the  narrow  cavity  approximation.  The  agreement  is  excellent 
for  the  curve  below  the  first  band  gap.  The  anomalies  above  may  be  explained 
by  the  comparatively  short  time  series. 
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Figure  6:  Dispersion  test  for  periodic  MILO  geometry.  The  contours  are  of 
amplitude  normalised  structure  factor  S{k,l)  for  Er  obtained  from  the  run 
mlp903,  whereas  the  graph  of  dispersion  curves  is  obtained  analytically.  Note 
the  origin  of  wavenumber  k  for  the  contours  is  at  the  centre  of  the  plot,  not  at 
the  left  as  implied  by  the  labelling. 
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5.6  Particle  Orbit  Tests  porb21,  porb35  and  porb40 

This  Section  demonstrates  the  ability  of  PIC3D  to  model  particle  orbits  in  a 
complicated  geometry,  essentially  that  of  une51  which  hcis  all  possible  inter¬ 
block  connectivities.  For  convenience,  each  cube  is  now  taken  to  have  side  1  cm 
and  a  4  X  4  X  4  meshing.  The  orbit  tests  involve  runs  of  400  timesteps  with 
At  =  4.811  X  10“^^s.  No  electric  field  is  present,  and  the  magnetic  field  has  only 
one  component  with  a  uniform  value  of  0.054  T,  chosen  so  that  the  electron 
gyro-radius  is  \/5/2  «  1.118  cm  for  an  orbital  speed  of  c/3  a  1  x  10®  m/s. 


SCATTER  PLOT 


Figure  7:  Plot  of  particle  position  projected  onto  the  plane  normal  to  B  {yz- 
plane),  in  the  cluster  of  eight  cubes  for  each  of  the  first  400  timesteps  of  run 
porb40. 


It  will  be  noted  that  the  diameter  of  the  gyro-orbit  exceeds  the  2  cm  dimen¬ 
sion  of  the  cube  cluster,  so  that  in  the  orbital  plane  each  inter-block  connection 
is  sampled.  The  result  in  plotting  space  is  an  indented  circular  orbit,  shown 
in  Fig  7,  which  exhibits  the  ability  of  PIC3D  to  generate  particle  scatter  plots 
directly  (such  files  are  suffixed  .gst,  and  the  figure  represents  a  collage  of  all 
the  frames  in  such  a  file).  The  basic  orbit  shown  is  clockwise,  but  the  indents 
appear  to  be  traversed  anticlockwise.  Imposing  an  initial  velocity  with  an  extra 
component,  parallel  to  the  applied  field,  ensures  that  every  normal  face  is  sam¬ 
pled,  and  hence  in  Fig  7  that  one  and  a  half  copies  of  the  basic  orbital  pattern 
appear. 

The  run  porb40  is  for  the  case  where  only  Bx  0,  and  the  centring  of  the 
orbit  implies  that  a  component  of  velocity  is  zero  at  some  boundaries.  Moving 
the  orbit  centre  by  an  amount  of  order  0.1  cm  leads  to  a  more  complete  test, 
exemplified  by  Fig  8  for  the  case  where  only  B^  ^  0  and  Fig  9  for  By  /  0.  In 
each  case  the  orbital  period  is  P  =  7.01  x  10“^®s,  so  that  2.75  cycles  are  covered 
in  a  run.  It  is  also  arranged  so  that  the  particle  travels  about  2.9  cm  parallel 
to  B.  Exact  details  may  be  obtained  from  the  “.uif”  listed  in  Appendix  A.  We 
see  that  in  all  cases  the  orbits  are  closed  and  have  the  expected  radius. 
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SCATTER  PLOT 
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Figure  8:  Plot  of  particle  position  projected  onto  the  plane  normal  to  B  {xy- 
plane),  in  the  cluster  of  eight  cubes  for  each  of  the  first  400  timesteps  of  run 
porb21. 


SCATTER  PLOT 


turn  .  CKf  4,  3>«<  <  >  f,P^ 


Figure  9:  Plot  of  particle  position  projected  onto  the  plane  normal  to  B  (xz- 
plane),  in  the  cluster  of  eight  cubes  for  each  of  the  first  400  timesteps  of  run 
porb35. 
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5.7  Orbit  Tests  porgSOh  and  porg70  in  General  Geometry 

There  are  essentially  two  new  geometry  types  to  test,  since  orthogonal  coordi¬ 
nate  systems  were  covered  in  the  previous  Section.  The  first  is  the  generalised 
cylindrical  geometry  shown  in  Fig  10.  The  central  region  is  gridded  by  three 
right  cylinders  which  are  parallelograms  in  cross-section.  Between  the  paral¬ 
lelograms  and  the  bounding  circle  of  radius  1  mm  are  six  blocks  where  the  e, 
lying  in  the  cross-section  plane  have  spatial  dependence.  The  total  number  of 
distinct  nodes  in  the  cross-section  is  132  and  there  are  6  elements  in  the  axial 
direction.  The  magnetic  field  is  2.16  T,  chosen  so  that  a  particle  with  an  initial 
speed  of  10®  m/s  has  a  gyro-orbit  of  diameter  0.559  mm.  The  subcycling  pa¬ 
rameter  Ns  =  10,  thus  for  a  Courant  limited  time-step  of  2.083  x  10“^®  s,  there 
are  8.4  particle  moves  per  orbit. 


Figure  10:  The  meshing  for  the  particle  orbit  test  porg60h. 

As  expected,  particle  orbits  confined  to  the  parallelogram  meshed  regions 
are  closed  and  repeat  exactly.  Such  a  test  is  a  severe  examination  not  only 
of  particle  motion  within  blocks,  but  also  of  the  transformations  applied  at 
block  boundaries.  Unfortunately,  the  situation  for  orbits  that  cross  the  outer 
blocks  is  less  clear  cut,  Figure  11  was  produced  from  a  .pts  output  file  using 
GNUPLOT.  It  shows  two  numerically  computed  orbits  for  the  run  porgSOh, 
the  upper  starting  at  a  radius  r  =  0.2795  mm,  on  the  y  axis  with  a  velocity  of 
10®  m/s  directed  only  in  x,  such  as  to  cause  outwards  travel.  The  lower  starts 
at  r  =  0.75  mm  on  the  y  axis  with  a  similar  velocity,  but  directed  so  as  to 
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travel  inwards.  Although  the  upper  orbit  is  not  closed,  throughout  the  run  of 
400  time-steps  or  nearly  50  orbits,  it  remains  a  satisfactory  approximation  to 
the  expected  motion.  The  lower  orbit  clearly  precesses  by  a  significant  amount. 
Other  similar  runs  show  no  obvious  pattern  in  the  way  some  orbits  process, 
whilst  others  remain  relatively  well  behaved. 

0.001 


0.0005 


>.  0 


•0.0005 


-0.001 

-0.001  -0.0005  0  0.0005  0.001 

X 

Figure  11:  Plot  of  two  particle  orbits  each  projected  onto  the  plane  normal  to 
B  (a:y-plane),  in  the  threefold  symmetric  meshing  of  the  cylinder  used  by  the 
run  porg60h. 


-0.001  -0.0005  0  0.0005  0.001 

X 

Figure  12:  Plot  of  two  particle  orbits  each  projected  onto  the  plane  normal  to  B 
(xy-plane),  in  the  orthogonal  meshing  of  the  cylinder  used  by  the  run  porb96. 

Shown  in  Fig  12  are  results  for  run  porb96  which  employs  an  orthogonal 
cylindrical  geometry  with  a  12  x  12  mesh  in  cross-section.  Here  Ns  =  5  and 
because  of  the  smaller  mesh  size  near  the  origin,  this  actually  corresponds  to 
84  steps  per  orbit,  of  which  approximately  12  are  completed  in  1000  steps.  The 
random  appearance  of  precession  here  cannot  be  attributed  to  discontinuities 
in  the  e*  (since  they  are  continuous),  or  to  taking  too  coarse  a  time-step  and 
applying  ten  corrector  steps  instead  of  one  has  no  effect  either.  It  seems  that 
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it  is  primarily  a  consequence  of  the  piecewise  constant  approximation  used  for 
the  (x,y)  dependence  of  E,  which  results  in  an  effective  oc  r~^  over  a  cell 
radius.  Similar  spurious  spatial  dependences  within  a  cell  are  expected  for  the 
nonorthogonal  geometry,  and  presumably  lead  to  the  observed  precession. 

The  last  orbit  test  concerns  a  completely  general  geometry,  that  of  the 
parallelepiped  for  which  the  generating  vectors  are  given  by 


/  1  1  0  \ 
0  10 
\2  2  l) 


(110) 


where  the  units  are  millimetres.  The  total  volume  occupied  is  one  cubic  mil¬ 
limetre,  and  for  orbit  testing  it  is  convenient  to  impose  periodic  boundary  con¬ 
ditions. 

The  run  porgTO  used  a  10  x  10  x  10  mesh  and  a  time-step  of  3.514  x  10“^^  s, 
so  that  Ns  =  10  implies  50  points  in  a  gyro-orbit.  Taking  periodicity  into 
account.  Fig  13  shows  the  expected  closed  orbits.  We  conclude  that  PIC3D  can 
accurately  simulate  particle  motion  in  complex,  non-orthogonal  geometries. 
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Figure  13:  Plot  of  particle  position  projected  onto  the  plane  normal  to 
B  (xy-plane),  in  the  general  parallelepipedal  geometry  for  each  of  the  first 
400  timesteps  of  run  porgTO. 


5.8  Particle  Emission  ml076 

The  geometry  is  an  annular  Magnetically  Insulated  Transmission  Line  (MITL), 
sketched  in  Fig  14.  The  electromagnetic  boundary  conditions  represent  perfect 
conductors  on  A,  B,  C,  D  and  E,  with  particle  emission  on  D  only,  and  F 
is  a  segment  of  the  polar  axis.  The  TEM  mode  with  an  electric  potential 
amplitude  of  500  kV  is  imposed  at  A,  with  field  directed  to  pull  electrons  from 
D.  The  number  of  charges  per  superparticle  is  5  x  10®.  The  main  annular  grid 
is  15  x  12  X  80  (composed  of  three  120°  blocks).  At  =  1.247  x  10“^^s  and  no 
subcycling  was  used.  Fig  15  plots  particle  radius  r  against  2r.  The  distribution 


TYKB/28006/TN/2  3D  PIC  Calculations 


37 


of  particle  positions  agrees  qualitatively  with  that  found  in  the  corresponding 
two-dimensional  test  case,  exhibiting  magnetic  insulation. 


Figure  14:  The  cross-section  of  the  annular  MITL  used  for  the  test  calculation 
of  particle  emission.  The  device  is  axisymmetric  about  the  line  containing  F, 
and  the  dimensions  (in  cm)  are  A  =  E  =  3.75,  B  =  7.5,  C  =  20  and 
F  =  1.5. 


5.9  Particle  Beam  Injection  bean32 

The  geometry  is  a  circular  cylinder,  but  meshed  in  cross-section  as  shown  in 
Fig  16.  The  cylinder  radius  is  1  mm,  and  the  central  cartesian  block  is  chosen 
to  have  a  maximum  radial  extent  of  0.7  mm,  ie.  a  side  of  length  0.7^/2  mm. 
The  central  square  has  20  x  20  elements  in  cross-section,  with  sides  of  length 
approximately  0.05  mm.  The  device  has  a  length  of  16  mm  in  the  axial  (z-) 
direction,  split  up  into  four  equal  sections,  so  that  there  are  20  blocks  altogether. 
The  mesh  spacing  in  z  is  0.1  mm,  and  the  timestep  at  unit  Courant  number 
is  6.055  X  10“^^  s.  The  cylinder  is  taken  to  have  perfectly  conducting  walls, 
except  for  the  flat  wall  at  extreme  z  which  is  chosen  to  be  resistive  with  a 
surface  impedance  equal  to  that  of  free  space. 

A  beam  of  electrons  of  circular  cross-section  with  radius  0.3  mm  is  injected 
at  z  =  0.  The  beam  voltage  is  7.085  kV,  and  the  total  (absolute)  current  is 
ramped  linearly  up  to  0.21  A,  over  an  interval  of  100  timesteps  so  as  to  reduce 
transient  effects.  The  number  of  charges  per  superparticle  is  6000,  so  that  at 
peak  power,  an  average  of  0.583  particles  are  emitted  per  element  in  time  Atp. 
The  sub  cycling  parameter  Ng  =  5. 

Fig  17  plots  particle  radius  r  against  z  after  1000  timesteps.  The  distribution 
of  particle  positions  agrees  quantitatively  with  that  expected  from  the  universal 
beam  spreading  curve[17.  Chap.  3],  for  a  laminar  beam  influenced  only  by  its 
self  field  at  small  perveance  (dimensionless  K  =  5.27  x  10“^  in  this  case).  The 
discrepancy  between  the  curve  and  the  particle  plots  is  accounted  for  by  the 
failure  of  high  order  quadratures  to  treat  accurately  step  function  integrals, 
such  as  are  needed  for  elements  at  the  beam  edge.  The  result  is  a  reduction  in 
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SCATTER  PLOT 


(a)  PIC3D  output 


JRcc'T't'Cale 


*n<tl  MITL  tmmt  H—<i. 

17.  1S.AV  MAT*. 
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(b)  2-D  atxisymmetric 

Figure  15:  Scatter  plot  of  particle  radius  versus  axial  coordinate  for  the  annular 
MITL,  at  time  i  =  9.0  ns  in  the  steady  state  regime.  At  bottom  is  a  plot  of 
the  corresponding  steady  state  computed  using  a  2-D  axisymmetric  code. 
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Figure  16:  The  meshing  of  the  cross-section  of  the  circular  cylinder  used  for  the 
test  calculation  of  particle  beam  injection. 


the  effective  beam  current,  hence  less  spreading  of  the  beam. 

5.10  Four  Cavity  MILO  nil407 

The  geometry  is  an  annular  Magnetically  Insulated  Line  Oscillator  (MILO), 
sketched  at  top  left  and  bottom  of  Fig  18.  Basically  the  MILO  consists  of  the 
MITL  with  the  addition  of  annular  cavities  on  the  outside  that,  in  the  example, 
extend  to  a  radius  of  7.5cm  and  which  are  4cm  in  axial  extent  with  a  separation 
of  1cm.  The  same  cavity  segment  was  studied  in  the  inlp903  calculations.  The 
dataset  ml407.uif  represents  a  four-cavity  MILO,  with  particle  emission  from 
all  surfaces  of  the  central,  cylindrical  cathode. 

The  plots  of  point  values  of  field  against  time  show  oscillations  with  a  period 
of  about  a  nanosecond,  see  Fig  19,  although  the  signals  are  rather  noisy.  The 
timestep  used  was  At  =  2.338  x  10”^^s  (no  subcycling)  and  the  number  of 
charges  per  superparticle  is  6  x  10®.  The  run  was  continued  to  a  time  of  50  ns, 
when  saturation  of  the  signal  was  clearly  apparent.  The  discrepancies  in  Fig  19 
are  accounted  for  by  the  different  units  employed,  and  that  the  PIC3D  data  was 
collected  at  a  more  interior  point.  We  conclude  that  PIC3D  is  able  to  simulate 
electromagnetic  particle  effects  in  the  complex  geometries  found  in  experiments. 
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SCATTER  PLOT 
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Time  =  0.303  ns  Step  =  1000  boan32  J  W  Eastwood  Frome  51 

Figure  17:  Scatter  plot  of  particle  radius  versus  axial  coordinate  for  the  run 
beaii32  with  circular  beam  injection,  at  time  t  =  0.3  ns  in  the  steady  state 
regime.  The  line  graph  is  a  plot  of  the  universal  beam  spreading  curve  at  the 
corresponding  parameters,  computed  by  solving  a  simple  ordinary  differential 
equation. 
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Figure  18:  At  the  top  is  PEGGIE  output  showing  a  4-cavity  MILO  and  at 
the  bottom  is  a  sketch  of  the  cross-section  of  the  annular  MILO:  the  device  is 
axisymmetric  about  the  centre-line,  running  left-to-right,  of  the  shaded,  cathode 
region. 
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MAGNETICALLY  INSULATED  LINE  OSCILLATOR 


ml407  Rum  0»-0B— •»  I3i02:0a  Rraaaaaad:  07tA«ie7  0»— OB  — SB  e_ml4.e7 


(a)  PIC3D  output 


xn.llo4  ci-ose  't.est.  fox*  3  —  D  code 
Mafirnet-lc  Field  a.t.  S\j.x>f&ee  End 


mlloA  Runt  2a~0«-«e  12i20tSB  Pro«*«a*clt  oai«2tSa  11— OB-BS  tadplet.Qrd 


(b)  2-D  axisymmetric 


Figure  19:  Point  values  of  magnetic  field  component  He  in  A  —  turn  m~^ 
plotted  cis  a  function  of  time  t  in  ns  for  the  PIC3D  run  nil407.  At  bottom  is  the 
corresponding  plot,  only  field  given  in  units  of  mT,  from  a  2-D  axisymmetric 
code. 
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5.11  Helix  TWT  pcllrOl 

Section  2 


Date:  10—05—95  Time:  16:46:35  Run  ID:  pcIlrOl  Frame  number:  6 

Figure  20:  Section  of  a  helix  TWT,  showing  the  central  helix  and  part  of  the 
tapered  dielectric  support,  modelled  by  the  PIC3D  run  pcllrOl. 

Unlike  the  previous  tests,  this  was  realised  without  use  of  PEGGIE,  so  that 
that  only  “.dat”  file  is  listed  in  this  Report,  see  Appendix  B.  The  calculation 
simulates  a  loop  of  helix  which  is  excited  by  ‘pinging’  the  start  of  the  first  turn, 
in  a  manner  similar  to  that  employed  by  run  inlp903.  The  geometry  of  one  of 
the  turns  or  sections  of  the  TWT  is  shown  in  Fig  20.  Other  details  (except 
dimensions  which  have  been  deleted  to  preserve  proprietary  information)  of  the 
TWT  modelled  may  be  inferred  from  the  following  log  data: 


General  Run  Data 


Run  identification  string:  pcllrOl 
Remarks : 

2  turn  cold  loopback  test 


44 


J  W  Eastwood,  W  Arter,  N  J  Brealey,  R  W  Hockney 


Run  time:  4.000E-10  s 

Time  series  start  time:  O.OOOE+00  s 

Time  series  end  time:  l.OOOE-08  s 

Time  series  output  interval:  4.000E-12  s 

Snapshot  start  time:  O.OOOE+00  s 

Snapshot  end  time:  l.OOOE-08  s 

Snapshot  output  interval:  2.000E-10  s 

Dispersion  start  time:  O.OOOE+00  s 

Dispersion  end  time:  l.OOOE-08  s 

Dispersion  output  interval:  4.000E-12  s 


Section  Data 


Number  of  Sections:  3 


number 

TYPE 

LENGTH 

SNAP  RECORD 

ACTIVE  REFERENCE 

1 

HELIX 

*****£-03 

no  yes 

yes 

2 

HELIX 

*****£-03 

no  yes 

yes 

3 

LOOPBACK 

O.OOOE+00 

no  yes 

yes 

First  Active  Section: 

1 

Number  of  Active  Sections:  3 

number 

Peak  PPM  (T) 

PPM  Period  (m) 

PPM  Zero 

(m)  Uniform  B  (T) 

1 

0 . OOOE+00 

O.OOOE+00 

O.OOOE+00 

*****£-01 

2 

O.OOOE+00 

O.OOOE+00 

O.OOOE+00 

*♦***£-01 

3 

O.OOOE+00 

O.OOOE+00 

O.OOOE+00 

*****£-01 

Nominal 

losses  in  dB/m: 

number 

Body  Tube 

Helix 

Vane 

1 

0 . OOOE+00 

0 . OOOE+00 

O.OOOE+00 

2 

O.OOOE+00 

0 . OOOE+00 

O.OOOE+00 

3 

0 . OOOE+00 

0 . OOOE+00 

O.OOOE+00 

Section  Niimber:  1 

Section  Type:  HELIX 


Length : 

Body  tube  radius: 
Helix  pitch: 
Helix  radius: 
Helix  width: 
Helix  thickness: 
Ping  helix: 


*****£-03  m 
*****£-03  m 
*****£-03  m 
*****£-03  m 
*****£-04  m 
*****£-04  m 

YES 
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Supports ; 

3 

Support  angles : 

6.000E+01 

degrees 

Support  angles : 

-6.000E+01 

degrees 

Support  angles : 

1 . 800E+02 

degrees 

Support  Type: 

WEDGE 

Support  perm. : 

*****£+00 

Support  loss: 

*****£+00 

dB/m 

Support  heights: 

0 . OOOE+00 

m 

Support  heights: 

0 . OOOE+00 

m 

Support  chords : 

*****£-03  m 

Support  chords : 

*****£-04  m 

Support  chords : 

O.OOOE+00  m 

Support  chords : 

O.OOOE+00  m 

Vanes : 

3 

Vane  angles: 

O.OOOE+00 

degrees 

Vane  angles: 

1 . 200E+02 

degrees 

Vane  angles: 

-1.200E+02 

degrees 

Vane  Type: 

NONE 

Vane  loss: 

O.OOOE+00 

dB/m 

Vane  heights: 

O.OOOE+00 

m 

Vane  heights: 

O.OOOE+00 

m 

Vane  chords : 

O.OOOE+00 

m 

Vane  chords: 

O.OOOE+00 

m 

Vane  chords: 

O.OOOE+00 

m 

Vane  chords: 

O.OOOE+00 

m 

Section  Number: 

2 

Section  Type: 

HELIX 

Length : 

*****£-03  m 

Body  tube  radius: 

*****£-03  m 

Helix  pitch: 

*****£-03  m 

Helix  radius: 

*****£-03  m 

Helix  width: 

*****£-04  m 

Helix  thickness: 
Ping  helix: 

*****£-04  m 

Supports: 

3 

Support  angles : 

6.000E+01  degrees 

Support  angles : 

-6.000E+01  degrees 

Support  angles : 

1.800E+02  degrees 

Support  Type: 

WEDGE 

Support  perm. : 

*****£+00 

Support  loss: 

*****£+00  dB/m 

Support  heights: 

O.OOOE+00  m 

Support  heights: 

O.OOOE+00  m 

Support  chords : 

*****£-03  m 

Support  chords : 

♦****£-04  m 

Support  chords: 

O.OOOE+00  m 

Support  chords : 

O.OOOE+00  m 

Vanes : 

3 

Vane  angles: 

O.OOOE+00  degrees 

Vane  angles: 

1.200E+02  degrees 
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Vane  eingles; 
Vane  Type: 

Vane  loss: 

Vane  heights: 
Vane  heights: 
Vane  chords: 

Vane  chords : 

Vane  chords: 

Vane  chords: 

-1.200E+02  degrees 
NONE 

O.OOOE+00  dB/m 
O.OOOE+00  m 

O.OOOE+00  m 

O.OOOE+00  m 

0 . OOOE+00  m 

O.OOOE+00  m 

O.OOOE+00  m 

Section  Number; 
Section  Type: 

3 

LOOPBACK 

Twist : 

Invert : 

O.OOOE+00  degrees 

Beam  Data 

Device  will  be 

empty  initially 

Number  of  Intervals :  1 

Radial  Interval  (m) 

Radial  Momenta  (Me) 

[  0 . OOOE+00 

O.OOOE+00] 

[  O.OOOE+00 

O.OOOE+00] 

Radial  Interval  (m) 

Azimuthal  Momenta  (Me) 

C  0 . OOOE+00 

O.OOOE+00] 

[  O.OOOE+00 

O.OOOE+00] 

Radial  Interval  (m) 

Axial  Momenta  (Me) 

[  O.OOOE+00 

O.OOOE+00] 

[  O.OOOE+00 

0 . OOOE+00] 

Radial  Interval  (m) 

Current  (A/m*’i‘2) 

[  O.OOOE+00 

O.OOOE+00] 

[  0 . OOOE+00 

0 . OOOE+00] 

Radial  Interval  (m) 

Particle  Weight 

[  O.OOOE+00 

O.OOOE+00] 

[  O.OOOE+00 

O.OOOE+00] 

Current  History  Data 


Form 


Form 


Form 


Form 


Form 


Number  of  Intervals :  1 
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Duration  (s) 
O.OOOE+00 
Duration  (s) 


O.OOOE+00 


Current  (A) 

[  O.OOOE+00  O.OOOE+00] 
RF  Current  (A) 

[  O.OOOE+00  O.OOOE+00] 


Signal  Data 


Number  of  Intervals: 
Zc (input) : 


Mesb  (Nominal)  Data 


Nominal  current: 
Nominal  beam  radius: 
Nominal  frequency: 
Nominal  momentum  (Me) : 
Nominal  helix  pitch: 
Nominal  delta  r: 
Nominal  delta  z: 
Nominal  delta  theta: 
Particles  per  element: 
Particles  width: 
Particles  height: 
Particles  theta  size: 


0 

0 .  ohms 


:(<***XcE-01  A 
****>i'E~04  m 
5.000E+09  Hz 

*****£-03  m 
****+£— 04  m 
****♦£— 04  m 
1.500E+01  degrees 
O.OOOE+00 
*****£-04  m 
*****£-04  m 
O.OOOE+00  degrees 


This  example  takes  about  20  minutes  to  run  on  a  Sun  SPARCstation  20, 
and  produces  the  following  files: 

pcllrOl-Ol.grl  which  contains  plots  of  fluxes  (through  planes  perpendic¬ 
ular  to  the  z  axis)  against  distance  along  the  z  axis  at  the  “snapshot”  times. 
The  quantities  plotted  are  total  displacement  flux  fDdS,  total  magnetic  flux 
/  B  •  dS,  and  current  /  j  •  dS. 

pcllr01-02.1in  which  is  intended  to  be  read  by  the  dispersion  analysis 
postprocessor  DISPAN.  It  contains  the  voltage  between  the  outer  surface  of  the 
helix  and  the  body  tube  at  d  =  0  as  a  function  of  axial  distance,  recorded  at 
the  “dispersion”  times. 

pcllrOl-OS.grl  which  contains  plots  of  the  field  components  Br,  Eq,  E~, 
Hr,  He,  Hz,  on  lines  parallel  to  the  z  axis  at  three  different  radii  -  half  an 
element  away  from  the  axis,  approximately  half  way  between  the  axis  and  the 
helix,  and  approximately  half  way  between  the  helix  and  body  tube.  The  plots 
are  produced  at  the  snapshot  times. 

pcllr01-04.tsd.  in  the  TSD  format  that  can  be  read  by  MPICTIM.  This 
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file  contains  global  energy  and  particle  diagnostics.  It  contains  the  total  num¬ 
ber  of  particles,  the  total  electric  energy,  the  total  magnetic  energy,  the  total 
electromagnetic  energy  and  the  total  particle  power.  The  data  is  collected  at 
the  “time  series”  times. 

pcllrOl-OB.gst  which  normally  contains  particle  scatter  plots  but  only 
contains  empty  plots  in  this  case  as  there  are  no  particles. 

pcllr01-06.1in  which  is  intended  to  be  read  by  the  dispersion  analysis 
postprocessor  DISPAN.  It  contains  plots  of  the  a:x;ial  electric  field  Ez  on  lines 
parallel  to  the  2  axis  at  two  different  radii  -  half  an  element  away  from  the 
axis  and  approximately  half  way  between  the  axis  and  the  helix.  The  output  is 
produced  at  the  dispersion  times. 

pcllrOl-OT.grl  is  not  produced  by  this  run  because  the  snapshot  (SNAP) 
variable  is  not  set  to  'YES'  in  any  of  the  TWT  sections.  It  would  normally 
contain  plots  of  field  components  against  radius  along  the  line  0  =  0  at  the 
start  of  each  section. 

pcllrOl-OS.gst  are  not  produced  by  this  run  because  the  SNAP  variable 
is  not  set  to  'YES '  in  any  of  the  TWT  sections.  It  normally  contains  particle 
scatter  plots. 

pcllr01-09.tsd  in  the  TSD  format.  It  contains  values  of  the  cylindrical 
polar  components  of  E  and  H  in  the  plane  at  the  start  of  each  TWT  section, 
at  points  near  the  axis,  half  way  between  the  axis  and  the  helix  and  half  way 
between  the  helix  and  the  body  tube.  The  data  is  collected  at  the  time  series 
times. 

pcllrOl-lO.tsd  in  the  TSD  format.  It  contains  the  particle  current  fiuxes 
through  plane  normal  to  the  2-axis  at  the  start  of  each  TWT  section.  These 
values  will  all  be  zero  in  this  case.  The  data  is  collected  at  the  time  series  times. 

pcllrOl.out  which  contains  run  diagnostic  data.  It  may  contain  error 
messages  if  a  run  stops  prematurely.  It  also  records  the  length  of  time  the  run 
took  to  complete,  the  name  of  the  workstation  the  code  was  run  on  and  the 
name  and  version  of  the  operating  system  it  was  using. 

Clearly  the  above  files  represent  an  enormous  amount  of  information  con¬ 
cerning  the  outcome  of  the  TWT  modelling  process.  The  most  important  data 
are  perhaps  the  tube  dispersion  properties.  The  first  .  lin  file  above  may  be 
used,  in  the  same  way  as  for  the  run  mlp903,  to  produce  a  contour  plot  of 
structure  factor,  where  the  locations  of  the  peaks  give  points  on  the  dispersion 
curve.  In  this  instance  it  proved  necessary  to  make  the  run  pcllr02,  which  has 
the  number  of  helix  turns  increased  to  five,  when  a  successful  comparison  was 
possible  with  (proprietary)  experimental  data,  see  Fig  21. 
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Figure  21:  Dispersion  test  for  periodic  TWT  geometry.  The  contours  are  of 
amplitude  normalised  structure  factor  S{k,l)  obtained  from  the  run  pcllr02. 
(The  axis  labelling  has  been  deliberately  removed  to  protect  proprietary  data.) 
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Appendices 

A  PEGGIE  test  data  examples 


bean32.mf 


o_bean3  2 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
bean32 

beam  spreading  test 


CHLAB3 

which  will 

appear  at  the  start 

CHLAB4 

of 

the  NOUT  channel  output 

' CHLAB5 

A 

**Label 

available  to  programmer 

1.1 

/ 

'NDIARY 

I 

**Channel  for  diary 

1.2 

/ 

'WIN 

I 

** Cur rent  input  channel 

1.2 

/ 

'NLEDGE 

I 

**Channel  for  restart  records 

1.2 

/ 

'NONLIN 

I 

♦♦Channel  for  input-output 

1.2 

/ 

'NOUT 

I 

♦♦Current  output  channel 

1.2 

/ 

'NPRINT 

I 

♦♦Channel  for  printed  output 

1.2 

/ 

'NPUNCH 

I 

♦♦Channel  for  card  output  (or  equivalent) 

1.2 

/ 

'NREC 

I 

♦♦Current  record  number 

1.2 

/ 

'NRUN 

I 

♦♦Maximum  number  of  steps 

1.2 

/ 

'NADUMP 

lA 

♦♦Codes 

for  array  dumps 

1.9 

/ 

'NPDUMP 

lA 

♦♦Codes 

for  dumping  points 

1.9 

/ 

'NVDUMP 

lA 

♦♦Codes 

for  dumping  arrays 

1.9 

/ 

'NLCHED 

L 

♦♦.TRUE. 

if  class  0  report-head  required 

1.9 

/ 

'NLHEAD 

LA 

♦♦.TRUE. 

,  if  class  1-9  report-heads  required 

1.9 

/ 

'NLOMTl 

LA 

♦♦Class 

1  subprogram  selector 

1.9 

/ 

'NLOMT2 

LA 

♦♦Class 

2  subprogram  selector 

1.9 

/ 

'NLOMT3 

LA 

♦♦Class 

3  subprogram  selector 

1.9 

/ 

'NLREPT 

L 

♦ ♦ . TRUE . 

if  report  required 

1.9 

/ 

'NLGRAF 

L 

♦♦display  device  geometry 

7.3 

F/ 

SF  courant  number 
&SF  DTMUL=1.0, 

NRUN=1000, 

CBLGLB= 'bl ' ,XYZGLB=-0. 0004949745, -0. 0004949745, 0 . , 

NLDUMP=F, 

NXPTDD=5/ 

PS  gst  pclesO  alia  timesl 
FS  pclesO  pplotO 
FL  pplotO 

&FL  CFLNAM= 'y (x) ' , LFSKIP=10 , LFCOLO=3 , FXMIN=-0 . 001 , FXMAX=0 . 001 , FMIN=-0 , 001 , FMAX=0 .001/ 

PS  gst  pclesl  alia  timesl 
FS  pclesl  pplotl 
FL  pplotl 

&FL  CFLNAM= ' r ( Z ) ' , LFSKIP=10 , LFCOLO=3 , FXMIN=0 . 0 , FXMAX=0 . 016 , FMIN=0 . 000 , FMAX=0 .001/ 

DS  alia  alldo 

DO  alldo  bis  b4s  b3s  b2E  b5s  bias  b4as  b3as  b2as  b5as  bibs  b4bs  b3bs  b2bs  b5bs  blcs  b4cs  b3cs  b2cs  bScs 

SU  bis  bl 

SU  b4s  b4 

SU  b3s  b3 

SU  b2s  b2 

SU  b5s  b5 

SU  bias  bla 

SU  b4as  b4a 

SU  b3as  b3a 

SU  b2as  b2a 

SU  b5as  bSa 

SU  bibs  bib 

SU  b4bs  b4b 

SU  b3bs  b3b 

SU  b2bs  b2b 

SU  b5bs  b5b 

SU  blcs  blc 

SU  b4cs  b4c 

SU  b3cs  b3c 

SU  b2cs  b2c 

SU  bScs  b5c 

TS  timesl  online 

BG  part2  polar_to_rectangular_transition  -8  -8  -6 

&POLRCT  RADCUR=0.001,RADSTR=0.0007,THEMIN=30. ,THEMAX=120. ,AXMIN=0. , AXMAX=0 . 004/ 

BG  cube  regular_cubic_lattice  -20  -20  -40 

&CUBREG  RZMAX=0 . 004 , RXMAX=0 . 000989949 , RYMAX=0 , 000989949/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1./ 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  cube  uniforml 
BL  b2  part2  uniforml 
BL  b3  sameas  b2 
BL  b4  sameas  b2 
BL  b5  sameas  b2 
BL  bla  sameas  bl 


bean32.uif 


BL 

b2a 

sameas 

b2 

BL 

b3a 

sameas 

b2 

BL 

b4a 

sameas 

b2 

BL 

b5a 

sameas 

b2 

BL 

bib 

sameas 

bl 

BL 

b2b 

sameas 

b2 

BL 

b3b 

sameas 

b2 

BL 

b4b 

sameas 

b2 

BL 

b5b 

sameas 

b2 

BL 

blc 

sameas 

bl 

BL 

b2c 

sameas 

b2 

BL 

b3c 

sameas 

b2 

BL 

b4c 

sameas 

b2 

BL 

b5c 

sameas 

b2 

PA  pbcon  pbinj  pcond 
SP  electrons  extra  1-16345 
&PCLE  NPBGEO=l/ 

PP  pbinj  inj  ect_beaiii 

&PARBCS  CSPECI=' electrons' , VBEAM=7 . 085E+3 , 
RADBEA  ( 2  )  =  0 . 0  0  0  3  ,  ROTBEA=  0  .  ,  CURBEA=  -0.21/ 
SG  inject  rainp2  signalcpt2 
SC  rainp2 
&SIGNAL 

FREQUENCY  =  0.0, 

DURATION  =  6.05E-12, 

POWER  =  0.0  1.0, 

SIGNAL_FORM= ' LINEAR ' / 

SC  signalcpt2 
&SIGNAL 

FREQUENCY  =  0.0, 

DURATION  =  10000., 

POWER  =  1.0  1.0, 

SIGNAL_FORM=  '  CONSTANT  '  / 

PP  pcond  perfect_conductor 
PA  pcond  sameas  pcond 
PP  reswal  resistive_wall 
&BCS  SURFZ=377. ,STHETA=1. 0  / 

PA  reswal  sameas  reswal 

BC  b2 

W=  bl:E 

S=b5:N 

N=b3 : S 

E=pcond 

U=b2a:D 

D=pcond 

EN 

BC  b3 

W=bl:N(l,0) (0,1) 

S=b2 :N 

N=b4 ; S 

E=pcond 

U=b3a:D 

D=pcond 

EN 

BC  b4 

W=bl:W(l,  0)  (0,1) 

S=b3 :N 

N=b5 : S 

E=pcond 

U=b4a:D 

D=pcond 

EN 

BC  b5 
W=bl : S 
S=b4 :N 
N=b2 : S 
E=pcond 
U=b5a:D 
D=pcond 
EN 

BC  bl 
S=b5:W 
E=b2 :W 

W(1,0) (0,l)=b4;W 
N(1,0)  (0,l)=b3:W 
D=pbcon 
U=bla:D 
EN 


bean32.mf 


BC  b2a 

W=  bla:E 

S=b5a:N 

N=b3a:S 

E=pcond 

U=b2b:D 


D=b2  :U 


EN 

BC  b3a 

W=bla:N(l, 0 

S=b2a:N 

N=b4a:S 

E=pcond 

U=b3b:D 

D=b3:U 


EN 

BC  b4a 
W=bla:W{l, 0) 
S=b3a:N 
N=b5a: S 
E=pcond 
U=b4b:D 
D=b4:U 


EN 

BC  bSa 

W=bla : S 

S=b4a:N 

N=b2a:S 

E=pcond 

U=b5b:D 

D=b5:U 

EN 

BC  bla 
S=b5a:W 
E=b2a:W 
W{1,0)  (0,  1) 
N(1,0)  (0,  1) 
D=bl:U 
U=blb : D 


EN 

BC  b2b 

W=  blb:E 

S=b5b:N 

N=b3b:S 

E=pcond 

U=b2c:D 

D=b2a:U 

EN 

BC  b3b 

W=blb:N(l,0) 

S=b2b:N 

N=b4b:S 

E=pcond 

U=b3c:D 

D=b3a:U 

EN 

BC  b4b 
W=blb:W(l, 0) 
S=b3b:N 
N=b5b:S 
E=pcond 
U=b4c : D 
D=b4a:U 
EN 

BC  bSb 

W=blb:S 

S=b4b:N 

N=b2b:S 

E=pcond 

U=b5c:D 

D=b5a:U 

EN 

BC  bib 
S=b5b:W 
E=b2b:W 
W(1,0)  (0,1)  = 
N(1,0)  (0,1)  = 
D=bla:U 


(0,1) 


:b4a :  W 
■b3a :  W 


(0,1) 


(0,1) 


:b4b:W 

:b3b:W 


bean32.mf 


EN 

BC  b2c 

W=  blc:E 

S=b5c:N 

N=b3c:S 

E=pcond 

U=reswal 

D=b2b;U 

EN 

BC  b3c 

W=blc:N(l,0) (0,1) 

S=b2c:N 

N=b4c:S 

E=pcond 

U=reswal 

D=b3b:U 

EN 

BC  b4c 

W=blc:W(l,0) (0,1) 

S=b3c:N 

N=b5c : S 

E=pcond 

U=reswal 

D=b4b:U 


EN 

BC  b5c 

W=blc : S 

S=b4c;N 

N=b2c:S 

E=pcond 

U=reswal 

D=b5b:U 

EN 

BC  blc 

S=b5c:W 

E=b2c:W 

W(1,0) (0,l)=b4c:W 
N(1,0) (0,l)=b3c:W 
D=blb:U 
U=reswal 


EN 

OR  bl  bla  bib  blc  b2  b3  b4  b5  b2a  b3a  b4a  b5a  b2b  b3b  b4b  bSb  b2c  b3c  b4c  bSc 


cyl07.uif 


o_cyl07 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
cyl07 

CYLINDRICAL  CAVITY 
UNIFORM  B  1  BLOCK  ONLY 
new  G33E 


'  CHLAB5 

A 

**Label 

available  to  programmer 

1.1 

'  / 

'NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

'NIN 

I 

**Current 

input  channel 

1.2 

/ 

'NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

'NONE  IN 

I 

** Channel 

for  input-output 

1.2 

/ 

'NOUT 

I 

**Current 

output  channel 

1.2 

/ 

'NPRINT 

I 

**Channel 

for  printed  output 

1.2 

/ 

'NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

'NREC 

I 

* ‘Current 

record  number 

1.2 

/ 

'NRUN 

I 

* ‘Maximum 

number  of  steps 

1.2 

/ 

'NADUMP 

lA 

“Codes 

for  array  dumps 

1.9 

/ 

'NPDUMP 

lA 

“Codes 

for  dumping  points 

1.9 

/ 

'NVDUMP 

lA 

“Codes 

for  dumping  arrays 

1.9 

/ 

'NLCHED 

L 

“.TRUE. 

if  class  0  report-head  required 

1.9 

/ 

'NLHEAD 

LA 

“.TRUE. 

,  if  class  1-9  report-heads  required 

1.9 

/ 

'NLOMTl 

LA 

“Class 

1 

subprogram  selector 

1.9 

/ 

'NLOMT2 

LA 

“Class 

2 

subprogram  selector 

1.9 

/ 

'NLOMT3 

LA 

“Class 

3 

subprogram  selector 

1.9 

/ 

'NLREPT 

L 

‘ ‘ . TRUE . 

if  report  required 

1.9 

/ 

'NLGRAF 

L 

“display 

device  geometry 

7.3 

F/ 

SF  courant  number 
&SF  DTMUL=1.0, 

NRUN=120, 

NXPTDD=5, 

NINIT=1 , 

BUNI(1)=1./ 

PS  tsd  ptflds  points  times 

FS  ptflds  El  E2  E3  HI  H2  H3 

DS  points  ptl  pt2  pt3  pt4  pt5 

SU  ptl  bl  1  2  3  0.0  0.0  0.0  0.0  0.0  0.0 

SU  pt2  bl  1  2  3  0.0  0.0  0.2  0.0  0.0  0.2 

SU  pt3  bl  1  2  3  0.0  0.0  0.6  0.0  0.0  0.6 

SU  pt4  bl  1  2  3  0.0  0.0  0.8  0.0  0.0  0.8 

SU  pt5  bl  1  2  3  0.0  0.0  1.0  0.0  0.0  1.0 

TS  times  online 

PS  tsd  avfld  face  times 

FS  avfld  intD.dS  intB.dS  intj.dS 

DS  face  onbl 

SU  onbl  bl  1  2  3  0.4  0.  0.  0.4  1.  1. 

BG  centre  polar_with_regular_meshing  -8  -8  -16 
&POLREG  RADINR=0. , RADOUT=0 . 001 , THEMAX=3 60 . , AXMAX=0 . 002 / 
BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1. / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  centre  uniforml 
PP  pcond  perfect_conductor 
PP  axis  polar_axis 
&BCS  / 

PA  pcond  sameas  pcond 
PA  axis  sameas  axis 
BC  bl 
E=pcond 

D(1,0)  (0,l)=pcond 

U(1,0) (0, l)=pcond 

S=bl:N 

N=bl : S 

W=axis 

EN 

OR  bl 
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o_mlp903 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
inlp903 

MAGNETICALLY  INSULATED  LINE  OSCILLATOR 

9  cavities 

periodic 

ping  limited  to  one  cavity 


'CHLAB5  A  **Label  available  to  programmer  1.1  '  / 

'NDIARY  I  * ‘Channel  for  diary  1.2  '  / 

'NIN  I  “Current  input  channel  1.2  '  / 

'NLEDGE  I  “Channel  for  restart  records  1.2  '  / 

'NONLIN  I  “Channel  for  input-output  1.2  '  / 

'NOUT  I  “Current  output  channel  1.2  '  / 

'NPRINT  I  “Channel  for  printed  output  1.2  '  / 

'NPUNCH  I  “Channel  for  card  output  (or  equivalent)  1.2  '  / 

'NREC  I  “Current  record  number  1.2  '  / 

'NRUN  I  “Maximum  number  of  steps  1.2  '  / 

'NADUMP  lA  “Codes  for  array  dumps  1.9  '  / 

'NPDUMP  lA  “Codes  for  dumping  points  1.9  '  / 

'NVDUMP  lA  “Codes  for  dximping  arrays  1.9  '  / 

'NLCHED  L  “.TRUE,  if  class  0  report-head  required  1.9  '  / 

'NLHEAD  LA  “.TRUE,  if  class  1-9  report-heads  required  1.9  '  / 

'NLOMTl  LA  “Class  1  subprogram  selector  1.9  '  / 

'NL0MT2  LA  “Class  2  subprogram  selector  1.9  '  / 

'NL0MT3  LA  “Class  3  subprogram  selector  1.9  '  / 

'NLREPT  L  “.TRUE,  if  report  required  1.9  '  / 

'NLGRAF  L  “display  device  geometry  7.3  '  F/ 


SF  courant  number 
&SF  DTMUL=0.99, 

NRUN=1053, 

NINIT=-9, 

CM0DE=1. , 

AMODE=0.15,-0.05,0.0, 

NLDUMP=F, 

NXPTDD=5/ 

PS  lin  Efield  outerline  timesp 
FS  Efield  El  E2  E3 
DS  outerline  outer 
CU  outer  inherit 
&INHERI  CBLOCK='b2a' , 

OPOINT=0.5,0.5,0. , 

NDIR{1)=3, 

NLXTEN=T , CUNIVS= 'all'/ 

TS  timesp  online  0.  5.  0.02  0.  0.  9 

TS  times  online 

PS  tsd  ptflds  points  times 

FS  ptflds  El  E2  E3  HI  H2  H3 

DS  points  ptl  pt2  pt3  pt4  pt5  pt6  pt7  pt8 

SU  ptl  ca2a  1  2  3  0.5  0.5  0.5  0.5  0.5  0.5 

SU  pt2  ca2b  1  2  3  0.5  0.5  0.5  0.5  0.5  0.5 

SU  pt3  ca2c  1  2  3  0.5  0.5  0.5  0.5  0.5  0.5 

SU  pt4  ca2  1  2  3  0.5  0.5  0.5  0.5  0.5  0.5 

TS  times  online 

PS  tsd  avfld  face  times 

FS  avfld  intD.dS  intB.dS  intj.dS 

DS  face  onb2 

SU  onb2  b2  1  2  3  0.4  0.  0.  0.4  1.  1. 

BG  centre  polar_with_regular_meshing  -8  -12  -10 
APOLREG  RADINR=0. , RADOUT=0 . 0375 , THEMAX=3 60 . , AXMAX=0 . 025 / 

BG  outer  polar_with_regular_meshing  -8  -4  -16 

&POLREG  RADINR=0.0375,RADOUT=0.075,THEMAX=120. ,AXMAX=0.04/ 

BG  cavity  polar_with_regular_meshing  -16  -4  -16 

StPOLREG  RADINR=0.075,RADOUT=0.150,THEMAX=120. ,AXMAX=0.04/ 

BG  inout  polar_with_regular_meshing  -8  -4  -20 

&POLREG  RADINR=0.0375,RADOUT=0.075,THEMAX=120. ,AXMAX=0.05/ 

BG  first  polar_with_regular_meshing  -8  -4  -4 

&POLREG  RADINR= 0 .0375, RADOUT=  0.075, THEMAX= 120., AXMAX=  0.01/ 

BP  uniforml  uniform 

&UNIFRM  EPSR=1 . ,RMUR=1. / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  b2  outer  uniforml 
BL  b3  sameas  b2 
BL  b4  sameas  b2 
BL  ca2  cavity  uniforml 
BL  ca3  sameas  ca2 
BL  ca4  sameas  ca2 
BL  b2a  inout  uniforml 
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BL  b3a  sameas  b2a 
BL  b4a  sameas  b2a 
BL  ca2a  sameas  ca2 
BL  ca3a  sameas  ca2 
BL  ca4a  sameas  ca2 
BL  b2b  sameas  b2a 
BL  b3b  sameas  b2a 
BL  b4b  sameas  b2a 
BL  ca2b  sameas  ca2 
BL  ca3b  sameas  ca2 
BL  ca4b  sameas  ca2 
BL  b2c  sameas  b2a 
BL  b3c  sameas  b2a 
BL  b4c  sameas  b2a 
BL  ca2c  sameas  ca2 
BL  ca3c  sameas  ca2 
BL  ca4c  sameas  ca2 
BL  b2d  sameas  b2a 
BL  b3d  sameas  b2a 
BL  b4d  sameas  b2a 
BL  ca2d  sameas  ca2 
BL  ca3d  sameas  ca2 
BL  ca4d  sameas  ca2 
BL  b2e  sameas  b2a 
BL  b3e  sameas  b2a 
BL  b4e  sameas  b2a 
BL  ca2e  sameas  ca2 
BL  ca3e  sameas  ca2 
BL  ca4e  sameas  ca2 
BL  b2f  sameas  b2a 
BL  b3f  sameas  b2a 
BL  b4f  sameas  b2a 
BL  ca2f  sameas  ca2 
BL  ca3f  sameas  ca2 
BL  ca4f  sameas  ca2 
BL  b2g  sameas  b2a 
BL  b3g  sameas  b2a 
BL  b4g  sameas  b2a 
BL  ca2g  sameas  ca2 
BL  ca3g  sameas  ca2 
BL  ca4g  sameas  ca2 
BL  b2h  sameas  b2a 
BL  b3h  sameas  b2a 
BL  b4h  sameas  b2a 
BL  ca2h  sameas  ca2 
BL  ca3h  sameas  ca2 
BL  ca4h  sameas  ca2 
BL  b2E  first  uniforml 
BL  b3E  sameas  b2E 
BL  b4E  sameas  b2E 
PP  pcond  perf ect_conductor 
&BCS  / 

PP  surfld  surface_f ield 

&BCS  DAPLYA(1)=1. , DAPLYA ( 2 ) =0 . , DAPLYA(3 ) =0 . , 
SURFZ=377 . ,NGLOBC=0,DPOT=1000. / 

PA  pcond  sameas  pcond 

PA  surfld  sameas  surfld 

BC  b2 

W=pcond 

N=b3 : S 

S=b4:N 

E=ca2:W 

D=b2h;U 

Up  =b2E:D 

EN 

BC  b3 
W=pcond 
N=b4 : S 
S=b2:N 
E=ca3 :W 
D=b3h:U 
U=b3E;D 
EN 

BC  b4 
W=pcond 
N=b2 : S 
S=b3 :N 
E=ca4 : W 


IE 
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D=b4h:U 

U=b4E:D 

EN 

BC  ca2 
W=b2 : E 
N=ca3:S 
S=ca4 :N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  ca3 
W=b3 : E 
N=ca4 ; S 
S=ca2:N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  ca4 
W=b4 : E 
N=ca2 : S 
S=ca3:N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  b2a 
W=  pcond 
N=b3a:S 
S=b4a:N 

E(0,0) (l,0.8)=ca2a:W 
E{0,0.8) (1,1) =pcond 
D=b2E:U 
Up  =  b2b:D 
EN 

BC  b3a 
W=pcond 
N=b4a:S 
S=b2a:N 

E  (0, 0)  (1, 0. 8) =ca3a:W 

E(0,0.8) (1,1) =pcond 

D=b3E:U 

U=b3b:D 

EN 

BC  b4a 
W=pcond 
N=b2a:S 
S=b3a:N 

E(0,0) (l,0.8)=ca4a:W 

E(0,0.8) (1,1) =pcond 

D=b4E:U 

U=b4b:D 

EN 

BC  ca2a 

W=b2a:E(0,0) (1,0.8) 

N=ca3a: S 

S=ca4a:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3a 

W=b3a:E(0,0) (1,0.8) 

N=ca4a: S 

S=ca2a:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4a 

W=b4a:E(0, 0) (1,0.8) 

N=ca2a:S 

S=ca3a:N 

E=pcond 

D=pcond 

U=pcond 

EN 
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BC  b2b 
W=  pcond 
N=b3b:S 
S=b4b:N 
E(0, 0)  (1,0. 
E(0,0.8) (1, 
D=b2a:U 
Up  =  b2c:D 
EN 

BC  b3b 

W=pcond 

N=b4b:S 

S=b2b;N 

E(0,0) (1,0. 

E(0,0.8) (1, 

D=b3a:U 

U=b3c:D 

EN 

BC  b4b 

W=pcond 

N=b2b:S 

S=b3b:N 

E(0,0)  (1,0. 

E(0,0.8) (1, 

D=b4a:U 

U=b4c:D 

EN 

BC  ca2b 

W=b2b:E(0, 0 

N=ca3b:S 

S=ca4b:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3b 

W=b3b;E (0, 0 

N=ca4b:S 

S=ca2b:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4b 

W=b4b:E (0, 0 

N=ca2b:S 

S=ca3b:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2c 
W=  pcond 
N=b3c:S 
S=b4c:N 
E(0,0) (1,0. 
E(0,0.8) (1, 
D=b2b:U 
Up  =  b2d:D 
EN 

BC  b3c 

W=pcond 

N=b4c:S 

S=b2c:N 

E(0,0) (1,0. 

E(0, 0. 8)  (1, 

D=b3b:U 

U=b3d:D 

EN 

BC  b4c 

W=pcond 

N=b2c:S 

S=b3c:N 

E(0,0) (1,0. 

E(0,0.8) (1, 

D=b4b:U 

U=b4d:D 

EN 


8) =ca2b:W 
1 ) =pcond 


8) =ca3b:W 
1) =pcond 


8) =ca4b:W 
1) =pcond 


) (1,0.8) 


) (1,0.8) 


)  (1,0.8) 


8) =ca2c:W 
1 ) =pcond 


8) =ca3c:W 
1 ) =pcond 


8)  =ca4c:W 
1) =pcond 
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BC  ca2c 


W=b2c:E(0,0) (1,0.8) 

N=ca3c : S 

S=ca4c :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3c 

W=b3c:E(0,0) (1,0.8) 

N=ca4c : S 

S=ca2c:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4c 

W=b4c:E(0,0) (1,0.8) 

N=ca2c : S 

S=ca3c :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2d 
W=  pcond 
N=b3d:S 
S=b4d:N 

E(0,0) (l,0.8)=ca2d:W 
E(0,0.8) (1,1) =pcond 
D=b2c:U 
Up  =  b2e:D 
EN 

BC  b3d 
W=pcond 
N=b4d:S 
S=b2d:N 

E(0,0) (1, 0. 8)=ca3d:W 

E(0,0.8) (1,1) =pcond 

D=b3c:U 

U=b3e:D 

EN 

BC  b4d 
W=pcond 
N=b2d:S 
S=b3d:N 

E(0,0)  (l,0.8)=ca4d:W 

E(0,0.8)  (1,1) =pcond 

D=b4c:U 

U=b4e:D 

EN 

BC  ca2d 

W=b2d:E(0,0) (1,0.8) 

N=ca3d:S 

S=ca4d:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3d 

W=b3d:E(0,0) (1,0.8) 

N=ca4d: S 

S=ca2d:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4d 

W=b4d:E(0,0) (1,0.8) 

N=ca2d:S 

S=ca3d:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2e 
W=  pcond 
N=b3e:S 
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S=b4e:N 
E(0,0) (1,0.8) 
E(0,0.8) (1,1) 
D=b2d:U 
Up  =  b2f;D 
EN 

BC  b3e 
W=pcond 
N=b4e:S 
S=b2e:N 
E(0,0) (1,0.8) 
E(0,0.8) (1,1) 
D=b3d:U 
U=b3  f : D 
EN 

BC  b4e 
W=pcond 
N=b2e:S 
S=b3e:N 
E(0,0) (1,0.8) 
E(0,0.8)  (1,1) 
D=b4d:U 
U=b4f  :D 
EN 

BC  ca2e 
W=b2e:E(0,0)  ( 
N=ca3e: S 
S=ca4e:N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  ca3e 
W=b3e:E(0,0)  ( 
N=ca4e : S 
S=ca2e:N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  ca4e 

W=b4e:E(0,0) ( 

N=ca2e:S 

S=ca3e:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2f 
W=  pcond 
N=b3f :S 
S=b4f  :N 
E(0,0)  (1,0.8) 
E{0,0.8)  (1,1)^ 
D=b2e:U 
Up  =  b2g:D 
EN 

BC  b3f 
W=pcond 
N=b4f :S 
S=b2f :N 
E(0,0)  (1,0.8): 
E(0,0.8) (1,1) 
D=b3e:U 
U=b3g:D 
EN 

BC  b4f 
W=pcond 
N=b2f :S 
S=b3f :N 
E(0,0)  (1,0.8) 
E{0,0.8) (1,1) 
D=b4e:U 
U=b4g:D 
EN 

BC  ca2f 
W=b2f :E(0,0) ( 
N=ca3f :S 


=ca2e :  W 
=pcond 


=ca3e:W 

=pcond 


=ca4e :  W 
=pcond 


=ca2f  :W 
=pcond 


=ca3f  :W 
=pcond 


=ca4f  :W 
=pcond 
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S=ca4f :N 
E=pcond 
D=pcond 
U=pcond 


EN 


BC  ca3f 

W=b3f ;E(0, 0) (1,0.8) 

N=ca4f : S 

S=ca2f :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4f 

W=b4f :E(0, 0) (1,0.8) 

N=ca2f : S 

S=ca3f : N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2g 
W=  pcond 
N=b3g:S 
S=b4g:N 

E(0,0) (l,0.8)=ca2g:W 
E(0,0.8) (l,l)=pcond 
D=b2f :U 
Up  =  b2h:D 
EN 

BC  b3g 
W=pcond 
N=b4g:S 
S=b2g:N 

E(0, 0) (1, 0. 8)=ca3g:W 
E(  0,0.8)  (l,l)=pcond 
D=b3f :U 
U=b3]l:D 


EN 

BC  b4g 
W=pcond 
N=b2g:S 
S=b3g:N 

E(0,0) (l,0.8)=ca4g:W 
E(0,0.8) (l,l)=pcond 
D=b4f :U 
U=b4h:D 


EN 

BC  ca2g 

W=b2g:E(0, 0) (1,0.8) 

N=ca3g: S 

S=ca4g:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3g 

W=b3g:E(0,0) (1,0.8) 

N=ca4g:S 

S=ca2g:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4g 

W=b4g;E(0,0) (1,0.8) 

N=ca2g: S 

S=ca3g:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2h 
W=  pcond 
N=b3h:S 
S=b4h:N 

E(0,0) (l,0.8)=ca2h:W 
E(0,0.8)  (l,l)=pcond 


D=b2g:U 
Up  =  b2;D 


EN 

BC  b3h 


W=pcond 

N=b4h:S 


S=b2h:N 


E(0,0) (l,0.8)=ca3h:W 
E(0,0.8) (l,l)=pcond 
D=b3g:U 
U=b3 : D 
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EN 

BC  b4h 
W=pcond 
N=b2h:S 
S=b3h:N 

E(0,0) (l,0.8)=ca4h:W 

E(0,0.8) (l,l)=pcond 

D=b4g:U 

U=b4 : D 

EN 

BC  ca2h 

W=b2h:E(0,0) {1,0.8) 

N=ca3h:S 

S=ca4h:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3h 

W=b3h:E(0,0) (1,0.8) 

N=ca4h: S 

S=ca2h:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4h 

W=b4h:E(0,0) (1,0.8) 

N=ca2h:S 

S=ca3h:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2E 
W=  pcond 
N=b3E: S 
S=b4E:N 
E=pcond 
D=b2 : U 
Up  =  b2a:D 
EN 

BC  b3E 

W=pcond 

N=b4E : S 

S=b2E:N 

E=pcond 

D=b3:U 

U=b3a:D 

EN 

BC  b4E 

W=pcond 

N=b2E:S 

S=b3E:N 

E=pcond 

D=b4 : U 

U=b4a:D 

EN 

OR  b2  b3  b4  ca2  ca3  ca4  b2h  b3h  b4h  ca2h  ca3h  ca4h  b2g  b3g  b4g  ca2g  ca3g  ca4g  + 
b2f  b3f  b4f  ca2f  ca3f  ca4f  b2e  b3e  b4e  ca2e  ca3e  ca4e  + 
b2d  b3d  b4d  ca2d  ca3d  ca4d  + 

b2c  b3c  b4c  ca2c  ca3c  ca4c  b2b  b3b  b4b  ca2b  ca3b  ca4b  b2a  b3a  b4a  ca2a  ca3a  ca4a  b2E  b3E  b4E 
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MAGNETICALLY  INSULATED  TRANSMISSION  LINE 
Particle  emission 


CHLAB5 

A 

**Label 

available  to  programmer 

1.1 

/ 

NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

NIN 

I 

**Current 

input  channel 

1.2 

/ 

NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

NONLIN 

I 

** Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

**Current 

output  channel 

1.2 

/ 

NPRINT 

I 

** Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

**Current 

record  number 

1.2 

/ 

NRUN 

I 

*  *Maximum 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

**Codes 

for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

**Codes 

for  diimping  points 

1.9 

/ 

NVDUMP 

lA 

**Codes 

for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

*  * . TRUE . 

,  if  class  0  report-head  recguired 

1.9 

/ 

NLHEAD 

LA 

*  * . TRUE . 

if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

**Class 

1 

subprogram  selector 

1.9 

/ 

NLOMT2 

LA 

**Class 

2 

subprogram  selector 

1.9 

/ 

NLOMT3 

LA 

**Class 

3 

subprogram  selector 

1.9 

/ 

NLREPT 

L 

*  * . TRUE . 

,  if  report  required 

1.9 

/ 

NLGRAF 

L 

*  *display 

device  geometry 

7.3 

'  F/ 

SF  courant  nvunber 
&SF  DTMUL=0.99, 

NRUN=8021, 

NLDUMP=T , 

NXPTDD=5/ 

PS  gst  pcles  alia  times 
FS  pcles  pplotl  pplot2 
FL  pplotl 

&FL  CFLNAM='r(z) ' , LFSKIP=10 , LFC0L0=3 , FXMIN=-0 . 2 , FXMAX=0 . 05 , FMIN=0 . ,FMAX=0.075/ 

FL  pplot2 

&FL  CFLNAM= ' theta ( z ) ' , LFSKIP=10 , LFCOLO=4 , FXMIN=-0 . 2 , FXMAX=0 . 05 , FMIN=-180 . , FMAX=180. / 

DS  alia  alldo 

DO  alldo  bis  b2s  b3s  b4s 

SU  bis  bl 

SU  b2s  b2 

SU  b3s  b3 

SU  b4s  b4 

PS  tsd  ptflds  points  times2 

FS  ptflds  El  E2  E3  HI  H2  H3 

DS  points  ptl  pt2  pt3  pt4  pt5 

SU  ptl  b2  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 

SU  pt2  b2  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SU  pt3  b2  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  pt4  b2  1  2  3  0.5  0.5  0.8  0.5  0.5  0.8 

SU  pt5  b2  1  2  3  0.5  0.5  0.95  0.5  0.5  0.95 

TS  times  online  0  10  1  0  0  9 

TS  times2  online 

PS  grl  Efield  outerline  times 

FS  Efield  El  E2  E3 

DS  outerline  outer 

CU  outer  inherit 

ScINHERI  CBLOCK= '  b2  '  , 

OPOINT=0. ,0.  ,0.5, 

NDIR(1)=2, 

NLXTEN=T, CUNIVS= ' all ' / 

PS  tsd  avfld  face  times2 
FS  avfld  intD.dS  intB.dS  intj.dS 
DS  face  onb2 

SU  onb2  b2  1  2  3  0.4  0.  0.  0.4  1.  1. 

BG  centre  polar_with_regular_meshing  -15  -12  -6 
&POLREG  RADINR=0. , RADOUT=0 . 0375 , THEMAX=360 . , AXMAX=0 . 015/ 

BG  outer  polar_with_regular_meshing  -15  -4  -80 

&POLREG  RADINR=0. 0375, RADOUT=0. 075, THEMAX=120. ,AXMAX=0.2/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1./ 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  centre  uniforml 
BL  b2  outer  uniforml 
BL  b3  sameas  b2 
BL  b4  sameas  b2 
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PP  pcond  perf ect_conductor 
&BCS  / 

PP  reswal  resistive_wall 
&BCS  SURFZ= 3  77., STHETA= 1.0  / 

PP  aplfld  applied_f ield 

iiBCS  DAPLYA(1)=-1.  ,  DAPLYA  (2  )  =0  .  ,  DAPLYA  ( 3  )  =0  .  , 
DPOT=0 . 5E+6, SFORM= ' boundary ' / 

PP  axis  polar_axis 
&BCS  / 

SP  electrons  any  1-15981 
PP  emitter  emitter 

&PARBCS  CSPECI=' electrons' , EFFE=0 . 0 , ALFA=0 . 5/ 
PP  emitter2  emitter2 

&PARBCS  CSPECI=' electrons' , EFFE=1 . 0 , ALFA=0 . 5/ 

PA  pcond  sameas  pcond 

PA  reswal  sameas  reswal 

PA  aplfld  sameas  aplfld 

PA  axis  sameas  axis 

PA  pemcon  emitter  pcond 

PA  pemcon2  emitter2  pcond 

SG  boundary  ramp  signalcpt 

SC  ramp 

& SIGNAL 

FREQUENCY  =  0.0, 

DURATION  =  0.3E-9, 

POWER  =  0.0  1.0, 

SIGNAL_FORM= 'LINEAR ' / 

SC  signalcpt 
&SIGNAL 

DURATION  =  10000., 

POWER  =  1.0  1.0, 

SIGNAL_FORM= ' CONSTANT ' / 

BC  b2 

W(0,0) (1,0.05) =pcond 
W(0,0.05) (1 , 0 . 9 ) =pemcon 
W(0,0.9) {l,0.925)=pcond 

W{0, 0.925) (1,1)=  bl:E  (0,  0) (0 . 333333 , 1) 

N=b3 : S 
S=b4:N 
E=pcond 
D=aplf Id 
Up  =pcond 
EN 

BC  b3 

W(0,0) (1, 0.05)=pcond 
W(0,0.05) (1,0.9) =pemcon 
W(0,0.9) (l,0.925)=pcond 

W(0, 0.925)  {l,l)=bl:E(0. 333333,0)  (0.666667,1) 

N=b4 :  S 

S=b2 :N 

E=pcond 

D=aplf Id 

U=pcond 

EN 

BC  b4 

W(0,0) (1, 0.05)=pcond 

W(0,0.05) (1,0.9) =pemcon 

W(0,0.9) (l,0.925)=pcond 

W(0, 0.925)  (l,l)=bl:E (0.666667,0)  (1. ,1) 

N=b2 : S 

S=b3:N 

E=pcond 

D=aplf Id 

U=pcond 

EN 

BC  bl 

E(0,0) (0. 333333, l)=b2:W(0, 0.925) (1,1) 

E( 0.33 333 3,0) (0 . 666667 , 1) =b3 :W(0, 0 . 925 ) (1,1) 

E  (0.666667,0)  (1. , 1) =b4 ; W ( 0 , 0 . 925 )  (1,1) 

D=pcond 

U=pcond 

S=bl:N 

N=bl :  S 

W=axis 

EN 

OR  bl  b2  b3  b4 
W(0,0.9) (l,0.925)=pcond 
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MAGNETICALLY  INSULATED  LINE  OSCILLATOR 
4  cavities 
particle  emission 
DPOT  of  500  kV 


CHLAB5 

A 

♦♦Label 

available  to  programmer 

1.1 

/ 

NDIARY 

I 

♦♦Channel 

for  diary 

1.2 

/ 

NIN 

I 

♦♦Current 

input  channel 

1.2 

/ 

NLEDGE 

I 

♦♦Channel 

for  restart  records 

1.2 

/ 

NONLIN 

I 

** Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

♦♦Current 

output  channel 

1.2 

/ 

NPRINT 

I 

♦♦Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

♦♦Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

♦♦Current 

record  number 

1.2 

/ 

NRUN 

I 

♦♦Maximum 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

♦♦Codes 

for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

♦♦Codes 

for  dumping  points 

1.9 

/ 

NVDUMP 

lA 

♦♦Codes 

for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

♦  * . TRUE . 

.  if  class  0  report-head  required 

1.9 

/ 

NLHEAD 

LA 

♦♦.TRUE, 

.  if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

♦♦Class 

1 

subprogreim  selector 

1.9 

/ 

'NLOMT2 

LA 

♦♦Class 

2 

subprogram  selector 

1.9 

/ 

’NLOMT3 

LA 

♦♦Class 

3 

subprogram  selector 

1.9 

/ 

'NLREPT 

L 

♦♦ .TRUE 

if  report  required 

1.9 

/ 

'NLGRAF 

L 

♦♦display  device  geometry 

7.3 

F/ 

SF  courant  number 
&SF  DTMUL=0.99, 

NRUN=20000, 

NTPRES=1, 

NXPTDD=5/ 

PS  gst  pcles  alia  times 
FS  pcles  pplotl  pplot2 
FL  pplotl 

&FL  CFLNAM='r{z)  '  ,  LFSKIP=10 , LFCOLO=3 , FXMIN=-0 . 2 , FXMAX=0 . 05 , FMIN=0 . ,FMAX=0.150/ 

FL  pplot2 

&FL  CFLNAM= ' theta (z) ' , LFSKIP=10 , LFCOLO=4 , FXMIN=-0 . 2 , FXMAX=0 . 05 , FMIN=-180 . ,FMAX=180. / 
DS  alia  alldo 

DO  alldo  bis  b2s  b3s  b4s  ca2s  ca3s  ca4s  b2cs  b3cs  b4cs  ca2cs  ca3cs  ca4cs  + 

b2bs  b3bs  b4bs  ca2bs  ca3bs  ca4bs  b2as  b3as  b4as  ca2as  ca3as  ca4as  b2Es  b3Es  b4Es 

SU  bis  bl 

SU  b2s  b2 

SU  b3s  b3 

SU  b4s  b4 

SU  ca2s  ca2 

SU  ca3s  ca3 

SU  ca4s  ca4 

SU  b2cs  b2c 

SU  b3cs  b3c 

SU  b4cs  b4c 

SU  ca2cs  ca2c 

SU  ca3cs  ca3c 

SU  ca4cs  ca4c 

SU  b2bs  b2b 

SU  b3bs  b3b 

SU  b4bs  b4b 

SU  ca2bs  ca2b 

SU  ca3bs  ca3b 

SU  ca4bs  ca4b 

SU  b2as  b2a 

SU  b3as  b3a 

SU  b4as  b4a 

SU  ca2as  ca2a 

SU  ca3as  ca3a 

SU  ca4as  ca4a 

SU  b2Es  b2E 

SU  b3Es  b3E 

SU  b4Es  b4E 

PS  tsd  ptflds  points  times2 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  ptl  pt2  pt3  pt4  pt5 
SU  ptl  b2E  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 

SU  pt2  b2a  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  pt3  b2c  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SUpt4  b2  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 
SUpt5  b2  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 
PS  tsd  ptflds  cavs  times2 
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DS  cavs  cav2  cav3  cav4  cav5 

SU  cav2  ca2a  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav3  ca2b  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav4  ca2c  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav5  ca2  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 
TS  times  online 

TS  times2  online  0.  50.  0.006  0.  0.  9 
PS  grl  Efield  outerline  times 
FS  Efield  El  E2  E3 
DS  outerline  outer 
CU  outer  inherit 
{cINHERI  CBLOCK='b2b' , 

OPOINT=0.  ,  0.  ,0.8, 

NDIR(1)=2, 

NLXTEN=T, CUNIVS= ' all ' / 

PS  tsd  avfld  face  times2 
FS  avfld  intD.dS  intB.dS  intj.dS 
DS  face  onb2 

SU  onb2  b2  1  2  3  0.4  0.  0.  0.4  1.  1. 

BG  centre  polar_with_regular_meshing  -8  -12  -10 
&POLREG  RADINR=0. ,RADOUT=0. 0375, THEMAX=360. ,AXMAX=0. 025/ 

BG  outer  polar_with_regular_meshing  -8  -4  -16 

&POLREG  RADINR=0. 0375, RADOUT=0. 075, THEMAX=120. ,AXMAX=0.04/ 

BG  cavity  polar_with_regular_meshing  -16  -4  -16 
&POLREG  RADINR=0.075,RADOUT=0.150,THEMAX=120.  ,AXMAX=0.04/ 

BG  inout  polar_with_regular_meshing  -8  -4  -20 

&POLREG  RADINR=0 . 0375 , RADOUT=0 . 075 , THEMAX=12 0 . , AXMAX=0 . 05 / 

BG  first  polar_with_regular_meshing  -8  -4  -4 

&POLREG  RADINR=0 . 0375  ,  RADOUT=0 . 075  ,  THEMAX=120  .  ,  AXMAX=0 .01/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1. / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  centre  uniforml 
BL  b2  outer  uniforml 
BL  b3  sameas  b2 
BL  b4  sameas  b2 
BL  ca2  cavity  uniforml 
BL  ca3  sameas  ca2 
BL  ca4  sameas  ca2 
BL  b2a  inout  uniforml 
BL  b3a  sameas  b2a 
BL  b4a  sameas  b2a 
BL  ca2a  sameas  ca2 
BL  ca3a  sameas  ca2 
BL  ca4a  sameas  ca2 
BL  b2b  sameas  b2a 
BL  b3b  Soimeas  b2a 
BL  b4b  sameas  b2a 
BL  ca2b  sameas  ca2 
BL  ca3b  sameas  ca2 
BL  ca4b  sameas  ca2 
BL  b2c  sameas  b2a 
BL  b3c  sameas  b2a 
BL  b4c  sameas  b2a 
BL  ca2c  sameas  ca2 
BL  ca3c  sameas  ca2 
BL  ca4c  sameas  ca2 
BL  b2E  first  uniforml 
BL  b3E  sameas  b2E 
BL  b4E  sameas  b2E 
PP  pcond  perfect_conductor 
tBCS  / 

PP  reswal  resistive_wall 
&BCS  SURFZ=377. , STHETA=1.0  / 

PP  aplfld  applied_f ield 

&BCS  DAPLYA(1)=-1. ,DAPLYA(2)=0. ,DAPLYA(3)=0. , 

DPOT=0 . 5 E+ 6, SFORM= 'boundary' / 

PP  axis  polar_axis 
&BCS  / 

PA  pcond  sameas  pcond 
PA  reswal  sameas  reswal 
PA  aplfld  sameas  aplfld 
PA  axis  sameas  axis 
PA  pemcon  emitter  pcond 
SP  electrons  any  1-16981 
PP  emitter  emitter 

&PARBCS  CSPECI=' electrons' ,EFFE=1.0/ 
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SG  boundary  ramp  signalcpt 
SC  ramp 
& SIGNAL 

FREQUENCY  =  0.0, 

DURATION  =  0.3E-9, 

POWER  =  0.0,  1.0, 

SIGNAL_FORM=  '  LINEAR '  / 

SC  signalcpt 
&SIGNAL 

DURATION  =  10000. , 

POWER  =  1.0,  1.0, 

SIGNAL_FORM= ' CONSTANT ' / 

BC  b2 

W(0,0) ( 1, 0 . 375) =pemcon 

W(0, 0.375) (1,1)=  bl:E  (0,  0 ) ( 0 . 33 333 3 , 1 ) 

N=b3 : S 
S=b4;N 
E=ca2 :W 
D=b2c:U 
Up  =pcond 
EN 

BC  b3 

W{0,0) (1,0.375) =pemcon 

W(0, 0.375) (l,l)=bl:E{0. 333333,0) (0.666667,1) 

N=b4 : S 

S=b2:N 

E=ca3 : W 

D=b3c:U 

U=pcond 

EN 

BC  b4 

W(0,0) (1,0.375) =pemcon 

W(0, 0.375) (l,l)=bl:E(0. 666667, 0) (1. , 1) 

N=b2 : S 

S=b3:N 

E=ca4:W 

D=b4c:U 

U=pcond 

EN 

BC  bl 

E(0, 0) (0. 333333, l)=b2;W(0, 0.37 5) (1,1) 

E  (0. 333333, 0)  ( 0 . 666667 , 1 ) =b3 :W (0 , 0 . 375)  (1,1) 

E(0. 666667, 0) (1. , 1) =b4 :W (0 , 0 . 375 ) (1,1) 

D=pemcon 

U=pcond 

S=bl:N 

N=bl:S 

W=axis 

EN 

BC  ca2 
W=b2 : E 
N=ca3  :S 
S=ca4 :N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  ca3 
W=b3 : E 
N=ca4 :S 
S=ca2 :N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  ca4 
W=b4 : E 
N=ca2 : S 
S=ca3 :N 
E=pcond 
D=pcond 
U=pcond 
EN 

BC  b2a 
W=  pemcon 
N=b3a:S 
S=b4a:N 

E(0,0) (l,0.8)=ca2a:W 


E(0,0.8) (1,1) =pcond 
D=b2E:U 
Up  =  b2b:D 
EN 

BC  b3a 
W=pemcon 
N=b4a:S 
S=b2a:N 

E(0,0) (l,0.8)=ca3a:W 

E(0,0.8)(l,l) =pcond 

D=b3E:U 

U=b3b:D 

EN 

BC  b4a 
W=pemcon 
N=b2a:S 
S=b3a:N 

E(0,0) (l,0.8)=ca4a:W 

E(0,0. 8) (1, l)=pcond 

D=b4E:U 

U=b4b:D 

EN 

BC  ca2a 

W=b2a:E(0,0) (1,0.8) 

N=ca3a : S 

S=ca4a :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3a 

W=b3a:E(0,0) (1,0.8) 

N=ca4a ; S 

S=ca2a :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4a 

W=b4a:E(0,0) (1,0.8) 

N=ca2a : S 

S=ca3a :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2b 
W=  pemcon 
N=b3b:S 
S=b4b:N 

E(0,0) (1, 0.8)=ca2b:W 
E(0,0.8) (l,l)=pcond 
D=b2a:U 
Up  =  b2c:D 
EN 

BC  b3b 
W=pemcon 
N=b4b:S 
S=b2b:N 

E(0,0) (1, 0.8)=ca3b:W 

E(0,0.8) (l,l)=pcond 

D=b3a:U 

U=b3c:D 

EN 

BC  b4b 
W=peincon 
N=b2b:S 
S=b3b:N 

E(0,0) (l,0.8)=ca4b:W 

E(0,0.8) (l,l)=pcond 

D=b4a:U 

U=b4c:D 

EN 

BC  ca2b 

W=b2b:E(0,0) (1,0.8) 

N=ca3b:S 

S=ca4b :N 

E=pcond 
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D=pcond 

U=pcond 

EN 

BC  ca3b 

W=b3b:E(0,0) (1,0.8) 

N=ca4b:S 

S=ca2b:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4b 

W=b4b:E(0, 0) (1,0.8) 

N=ca2b:S 

S=ca3b:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2c 
W=  pemcon 
N=b3c:S 
S=b4c:N 

E(0,0) (l,0.8)=ca2c:W 
E(0,0.8) (l,l)=pcond 
D=b2b;U 
Up  =  b2:D 
EN 

BC  b3c 
W=pemcon 
N=b4c:S 
S=b2c:N 

E(0,0) (l,0.8)=ca3c:W 

E(0, 0.8) (l,l)=pcond 

D=b3b:U 

U=b3 : D 

EN 

BC  b4c 
W=pemcon 
N=b2c:S 
S=b3c:N 

E(0,0)  (l,0.8)=ca4c:W 

E(0,0.8) (l,l)=pcond 

D=b4b:U 

U=b4 : D 

EN 

BC  ca2c 

W=b2c:E(0,0) (1,0.8) 

N=ca3c : S 

S=ca4c :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca3c 

W=b3c:E(0,0) (1,0.8) 

N=ca4c : S 

S=ca2c :N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  ca4c 

W=b4c:E(0,0) (1,0.8) 

N=ca2c : S 

S=ca3c:N 

E=pcond 

D=pcond 

U=pcond 

EN 

BC  b2E 
W=  pcond 
N=b3E:S 
S=b4E:N 
E=pcond 
D=aplf Id 
Up  =  b2a:D 
EN 


ml407.uif 

BC  b3E 
W=pcond 
N=b4E:S 
S=b2E;N 
E=pcond 
D=aplf Id 
U=b3a:D 
EN 

BC  b4E 
W=pcond 
N=b2E:S 
S=b3E:N 
E=pcond 
D=aplfld 
U=b4a : D 
EN 

OR  bl  b2  b3  b4  ca2  ca3  ca4  b2c  b3c  b4c  ca2c  ca3c  ca4c  b2b  b3b  b4b  ca2b  ca3b  ca4b  b2a  b3a  b4a  ca2a  ca3a  ca4a  b2E 


porbll.uif 

^  r 

o_porb21 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 

porb21 

Uniform  B 

All  orientations  of  cubes 
particle  orbit  test 
z  cpt  of  velocity 


CHLAB5 

A 

**Label  available  to  programmer 

1.1 

/ 

NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

NIN 

I 

♦♦Current 

input  channel 

1.2 

/ 

NLEDGE 

I 

♦♦Channel 

for  restart  records 

1.2 

/ 

NONLIN 

I 

♦♦Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

♦ ♦Current 

output  channel 

1.2 

/ 

NPRINT 

I 

♦ ♦Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

♦♦Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

♦♦Current 

record  number 

1.2 

/ 

NRUN 

I 

♦ ♦Maximum 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

♦♦Codes  for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

♦♦Codes  for  dumping  points 

1.9 

/ 

NVDUMP 

lA 

♦♦Codes  for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

♦♦.TRUE.  , 

if  class  0  report-head  required 

1.9 

/ 

NLHEAD 

LA 

♦♦.TRUE. 

if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

♦♦Class  1 

subprogram  selector 

1.9 

/ 

NLOMT2 

LA 

♦♦Class  2 

subprogram  selector 

1.9 

/ 

NL0MT3 

LA 

♦♦Class  3 

subprogram  selector 

1.9 

/ 

NLREPT 

L 

♦♦ .TRUE. 

if  report  required 

1.9 

/ 

NLGRAF 

L 

♦♦display 

device  geometry 

7.3 

F/ 

SF  courant_number 
&SF  DTMUL=1.0, 

NRUN=400, 

NLDUMP=F, 

MALG(19)=1,MALG(20)=1, 

NXPTDD=5/ 

PS  gst  pclesO  alia  timesl 
FS  pclesO  pplotO 
FL  pplotO 

&FL  CFLNAM= 'y (x) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=-0 . 005 , FXMAX=0 . 025, FMIN=-0 . 01 , FMAX=0 . 02/ 
PS  gst  pclesl  alia  timesl 
FS  pclesl  pplotl 
FL  pplotl 

&FL  CFLNAM= 'X ( z ) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=0 . 0 , FXMAX=0 . 02 , FMIN=0 . 0 , FMAX=0 . 02 / 

DS  alia  alldo 

DO  alldo  bis  b2s  b3s  b4s  b5s  b6s  b7s  bSs 

SU  bis  bl 

SU  b2s  b2 

SU  b3s  b3 

SU  b4s  b4 

SU  b5s  b5 

SU  b6s  b6 

SU  b7s  b7 

SU  bSs  b8 

PS  tsd  ptflds  points  times2 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  pt2  pt3  pt4  pt5  cav3  cav4  cav5 
SU  pt2  b8  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SUpt3  b4  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SU  pt4  b2  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 

SU  pt5  b2  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  cav3  b7  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 
SU  cav4  b5  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 
SU  cav5  b3  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 
TS  timesl  online  0  1000  1000 
TS  times2  online  0  1000  1000 
BG  typel  regular_cubic_lattice  -4  -4  -4 
&CUBREG  RXMAX=0.01,RYMAX=0.01,RZMAX=0.01/ 

BG  type2  regular_cubic_lattice  -4  -4  -4 
&CUBREG  RXMAX=0.01,RYMAX=0.01,RZMAX=0.01/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1./ 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 
BL  b2  type2  uniforml 
BL  b3  sameas  bl 
BL  b4  sameas  bl 
BL  b5  sameas  bl 
BL  b6  sameas  bl 
BL  b7  sameas  bl 


porb21.uif 


BL  b8  sameas  bl 
SP  electrons  below  1 


-1 


0  0  8 


&PCLE  CBL0CK='bl', 
UCPOS=0.218,0.1, 0.5,  -1., 
VEL=0. ,-l.E+8,1.5E+7,NCVEL=l/ 
PO  bext 


1 


&SF  NINIT=1,BUNI=0. 0,0. 0,-0. 054/ 

PP  pcond  perf ect_conductor 
PP  reswal  resistive_wall 
&BCS  SURFZ=377. ,STHETA=0.5  / 

PP  aplfld  applied_f ield 

&BCS  DAPLYA (1)=115.5712, DAPLYA ( 2 ) =0 

PA  pcond  sameas  pcond 

PA  reswal  sameas  reswal 

PA  aplfld  sameas  aplfld 

BC  bl 


D=b5:N(l,0) (0,1) 
S=b4 :D 

W=b2:N(l,0) (0,1) 
E=b2:S(l,0) (0,1) 
N=b4:U 

U=b5:S(l,0) (0,1) 
EN 

BC  b2 


D=b6:U(0, 1)  (1,0) 
S(1,0) (0, l)=bl:E 
W=b3:S(0,l) (1,0) 
E=b3:N(0,l) (1,0) 
N(1,0)  (0, l)=bl:W 
U=b6:D(0, 1) (1,0) 
EN 

BC  b3 

D=b4:W(0,l) (1,0) 
S(0,1)  (l,0)=b2:W 
W=b7 : E 


E=b7:W 


N(0,1) (l,0)=b2:E 
U=b4:E(0, 1) (1,0) 
EN 

BC  b4 


D=bl : S 


S=b8:S (1, 0) (0,1) 
W(0,1) (l,0)=b3:D 
E(0,1) (l,0)=b3:U 
N=b8:N(l, 0) (0,1) 
U=bl:N 


EN 

BC  b5 

D=b8:D(0,l) (1,0) 
S(1,0)  (0,l)=bl:U 
W=b6 ; W 
E=b6 : E 

N(1,0) (0,l)=bl:D 
U=b8:U(0,l) (1,0) 
EN 

BC  b6 

D(0,1) (l,0)=b2:U 
S=b7:N(0, 1)  (1,0) 
W=b5:W 
E=b5:E 

N=b7:S(0,l) (1,0) 
U(0,1) (l,0)=b2:D 
EN 

BC  b7 

D=b8:E(l,l) (0,0) 
S(0,1)  (l,0)=b6:N 
W=b3 : E 
E=b3 : W 

N(0,1) (l,0)=b6:S 
U=b8:W(l, 1) (0,0) 
EN 

BC  b8 

D(0,1) (l,0)=b5:D 
S(1,0) (0,l)=b4:S 
W(l,l) (0, 0)=b7:U 
E(l,l)  (0,  0)=b7;D 
N(1,0) (0,l)=b4:N 
U(0,1) (l,0)=b5:U 


.  ,DAPLYA(3)=0.  / 


porb35.uif 


o_porb3  5 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 

porb35 

Uniform  B 

All  orientations  of  cubes 
particle  orbit  test 
B  in  y  direction 


'CHLAB5 

A 

** Label 

available  to  programmer 

1.1 

/ 

NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

NIN 

I 

** Cur rent 

input  channel 

1.2 

/ 

NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

NONLIN 

I 

** Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

**Current 

output  channel 

1.2 

/ 

NPRINT 

I 

** Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

**Current 

record  number 

1.2 

/ 

NRUN 

I 

**Maximum 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

**Codes 

for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

**Codes 

for  dumping  points 

1.9 

/ 

NVDUMP 

lA 

**Codes 

for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

**.TRUE, 

.  if  class  0  report-head  required 

1.9 

/ 

NLHEAD 

LA 

** .TRUE. 

.  if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

**Class 

1 

subprogram  selector 

1.9 

/ 

NLOMT2 

LA 

**Class 

2 

subprogram  selector 

1.9 

/ 

NLOMT3 

LA 

**Class 

3 

subprogram  selector 

1.9 

/ 

NLREPT 

L 

**.TRUE. 

,  if  report  required 

1.9 

/ 

NLGRAF 

L 

* ‘display 

device  geometry 

7.3 

F/ 

SF  courant_nmnber 
&SF  DTMUL=1.0, 

NRUN=400, 

NLDUMP=F, 

NINPOR ( 1 9 ) = 1 , NINPOR ( 2  0 ) = 1 , 

NXPTDD=5/ 

PS  gst  pclesO  alia  timesl 
FS  pclesO  pplotO 
FL  pplotO 

&FL  CFLNAM= ' z (x) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=0 . 0 , FXMAX=0 . 02 , FMIN=0 . 0 , FMAX=0 . 02/ 

PS  gst  pclesl  alia  timesl 
FS  pclesl  pplotl 
FL  pplotl 

&FL  CFLNAM= 'y (x) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=-0 . 005 , FXMAX=0 . 025 , FMIN=-0 . 01 , FMAX=0 . 02 / 
DS  alia  alldo 

DO  alldo  bis  b2s  b3s  b4s  b5s  b6s  b7s  bSs 

SU  bis  bl 

SU  b2s  b2 

SU  b3s  b3 

SU  b4s  b4 

SU  b5s  b5 

SU  b6s  b6 

SU  b7s  b7 

SU  bSs  b8 

PS  tsd  ptflds  points  times2 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  pt2  pt3  pt4  pt5  cav3  cav4  cav5 
SU  pt2  b8  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SUpt3  b4  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SU  pt4  b2  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 

SU  pt5  b2  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  cav3  b7  1  2  3  0.8  0.5  0.5  0.8  0.5  6.5 

SU  cav4  b5  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav5  b3  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

TS  timesl  online  0  1000  1  0  0  0 

TS  times2  online  0  1000  1000 
BG  typel  regular_cubic_lattice  -4  -4  -4 
&CUBREG  RXMAX=0.01,RYMAX=0.01,RZMAX=0.01/ 

BG  type2  regular_cubic_lattice  -4  -4  -4 
&CUBREG  RXMAX=0.01,RYMAX=0.01,RZMAX=0.01/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1. / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 
BL  b2  type2  uniforml 
BL  b3  sameas  bl 
BL  b4  sameas  bl 
BL  b5  sameas  bl 
BL  b6  sameas  bl 
BL  b7  sameas  bl 


porb35.mf 


BL  b8  sameas  bl 

SP  electrons  below  1-10081 
&PCLE  CBLOCK= ' bl ' , 

UCPOS=0. 218, 0.5, 0.95,  -1., 

VEL=0. ,1.5E+7, -l.E+8, 0. ,NCVEL=1/ 

PO  bext 

&SF  NINIT=1,BUNI=0.0,0.054,0.0/ 

PP  pcond  perfect_conductor 
PP  reswal  resistive_wall 
&BCS  SURFZ=377. ,STHETA=0.5  / 

PP  aplfld  applied_f ield 

&BCS  DAPLYA {1)=115.5712, DAPLYA ( 2 ) =0 . , DAPLYA ( 3 ) =0 . 
PA  pcond  sameas  pcond 
PA  reswal  sameas  reswal 
PA  aplfld  sameas  aplfld 
BC  bl 


D=b5:N(l,0) (0, 1) 
S=b4:D 


W=b2:N{l,0) (0,1) 
E=b2:S{l,0) (0,1) 
N=b4:U 


U=b5:S(l,0) (0,1) 
EN 

BC  b2 


D=b6;U(0, 1) (1,0) 
S(1,0) (0,l)=bl:E 
W=b3:S(0,l) (1,0) 
E=b3;N(0,l) (1,0) 
N(1,0) (0,l)=bl:W 
U=b6:D(0,l)  (1,0) 
EN 

BC  b3 


D=b4:W(0,l) (1,0) 
S(0,1) (l,0)=b2:W 
W=b7 : E 


E=b7:W 

N(0,1) (l,0)=b2:E 
U=b4:E(0,l) (1,0) 


EN 

BC  b4 


D=bl : S 

S=b8:S(l,0) (0,1) 
W(0,1) (l,0)=b3:D 
E(0,1) (l,0)=b3;U 
N=b8:N(l,0) (0,1) 
U=bl:N 


EN 

BC  b5 

D=b8:D(0,l) (1,0) 
S(1,0) (0,l)=bl:U 
W=b6:W 
E=b6:E 

N(1,0) (0,l)=bl:D 
U=b8:U(0,l) (1,0) 
EN 

BC  b6 

D(0,1) (l,0)=b2:U 
S=b7:N(0,l) (1,0) 
W=b5 :W 
E=b5:E 

N=b7:S(0,l) (1,0) 
U(0,1) (l,0)=b2:D 
EN 

BC  b7 

D=b8:E(l,l) (0,0) 
S(0,1) (1, 0)=b6:N 
W=b3 : E 
E=b3:W 

N(0,1)  (l,0)=b6:S 
U=b8:W(l, 1) (0,0) 
EN 

BC  b8 

D(0, 1)  (1, 0)=b5:D 
S(1,0) (0,l)=b4:S 
W(l,l) (0,0)=b7:U 
E(l,l) (0,0)=b7:D 
N{1,0) (0, l)=b4:N 
U(0,1) (1, 0)=b5:U 


/ 
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o_porb40 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 

porb40 

Uniform  B 

All  orientations  of  cubes 
particle  orbit  test 
field  in  x  direction 


CHLAB5 

A 

**Label  available  to  programmer 

1.1 

/ 

NDIARY 

1 

** Channel 

for  diary 

1.2 

/ 

NIN 

I 

**Current 

input  channel 

1.2 

/ 

NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

NONLIN 

1 

** Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

** Cur rent 

output  channel 

1.2 

/ 

NPRINT 

I 

** Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

** Cur rent 

record  number 

1.2 

/ 

NRUN 

I 

**Maximum 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

**Codes  for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

**Codes  for  dumping  points 

1.9 

/ 

NVDUMP 

lA 

**Codes  for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

**.TRUE. 

if  class  0  report-head  required 

1.9 

/ 

NLHEAD 

LA 

*  * . TRUE .  , 

if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

**Class  1 

subprogram  selector 

1.9 

/ 

NLOMT2 

LA 

**Class  2 

subprogram  selector 

1.9 

/ 

NLOMT3 

LA 

**Class  3 

subprogram  selector 

1.9 

/ 

NLREPT 

L 

**.TRUE. 

if  report  required 

1.9 

/ 

NLGRAF 

L 

**di splay 

device  geometry 

7.3 

F/ 

SF  courant_nmnber 
&SF  DTMUL=1.0, 

NRUN=400, 

NLDUMP=F, 

NINP0R(19) =1,NINPOR{20) =1, 

NXPTDD=5/ 

PS  gst  pclesO  alia  timesl 
FS  pclesO  pplotO 
FL  pplotO 

tFL  CFLNAM= 'y ( z )  ' , LFSKIP=1 , LFCOLO=3 , FXMIN=-0 . 005 , FXMAX=0 . 025 , FMIN=-0 . 01 ,  FMAX=0 . 02 / 
PS  gst  pclesl  alia  timesl 
FS  pclesl  pplotl 
FL  pplotl 

JcFL  CFLNAM=  '  X  ( Z )  '  ,  LFSKIP= 1 ,  LFCOLO=  3  ,  FXMIN=  0.0,  FXMAX=  0.02,  FMIN=  0.0,  FMAX=0 .02/ 

DS  alia  alldo 

DO  alldo  bis  b2s  b3s  b4s  b5s  b6s  b7s  bSs 

SU  bis  bl 

SU  b2s  b2 

SU  b3s  b3 

SU  b4s  b4 

SU  bSs  b5 

SU  b6s  b6 

SU  b7s  b7 

SU  b8s  b8 

PS  tsd  ptflds  points  times2 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  pt2  pt3  pt4  pt5  cav3  cav4  cav5 
SU  pt2  b8  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  pt3  b4  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SU  pt4  b2  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 

SU  pt5  b2  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  cav3  b7  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav4  b5  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav5  b3  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

TS  timesl  online  0  1000  1000 

TS  times2  online  0  1000  1000 

BG  typel  regular_cubic_lattic6  -4  -4  -4 
tCUBREG  RXMAX=0.01,RYMAX=0.01,RZMAX=0.01/ 

BG  type2  regular_cubic_lattice  -4  -4  -4 
&CUBREG  RXMAX=0.01,RYMAX=0.01,RZMAX=0.01/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1.  ,RMUR=1./ 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 
BL  b2  type2  uniforml 
BL  b3  sameas  bl 
BL  b4  sameas  bl 
BL  b5  sameas  bl 
BL  b6  sameas  bl 
BL  b7  sameas  bl 


porb40.uif 


BL  b8  sameas  bl 

SP  electrons  below  1-10081 
&PCLE  CBLOCK='bl', 

UCPOS=0. 5,1., 0.118,  -1., 

VEL=1.5E+7,-l.E+8,0. ,NCVEL=1/ 

PO  bext 

&SF  NINIT=1,BUNI=0.054,0.0,0.0/ 

PP  pcond  perf ect_conductor 
PP  reswal  resistive_wall 
&BCS  SURFZ=377. ,STHETA=0.5  / 

PP  aplfld  applied_f ield 

&BCS  DAPLYA(1)=115.5712,DAPLYA(2)=0. ,DAPLYA(3)=0.  / 
PA  pcond  saineas  pcond 
PA  reswal  saineas  reswal 
PA  aplfld  saineas  aplfld 
BC  bl 

D=b5:N{l,0) (0,1) 

S=b4 : D 

W=b2:N(l,0) (0,1) 

E=b2:S(l,0) (0,1) 

N=b4 :U 

U=b5:S(l,0) (0,1) 

EN 

BC  b2 

D=b6:U(0,l) (1,0) 

S(1,0) (0,l)=bl:E 
W=b3:S(0,l) (1,0) 

E=b3:N(0,l) (1,0) 

N(1,0) (0,l)=bl:W 
U=b6:D(0,l) (1,0) 

EN 

BC  b3 

D=b4:W(0,l) (1,0) 

S(0,1) (l,0)=b2;W 

W=b7 : E 

E=b7:W 

N(0,1) (l,0)=b2:E 
U=b4 :E(0,1)(1,0) 

EN 

BC  b4 
D=bl :  S 

S=b8:S(l,0) (0,1) 

W(0,1) (l,0)=b3:D 
E(0,1) (l,0)=b3:U 
N=b8 :N(1, 0) (0,1) 

U=bl:N 

EN 

BC  b5 

D=b8;D(0,l) (1,0) 

S(1,0) (0,l)=bl:U 
W=b6 :W 
E=b6 :E 

N(1,0) (0,l)=bl:D 
U=b8:U(0,l) (1,0) 

EN 

BC  b6 

D(0,1) (l,0)=b2:U 
S=b7:N(0,l) (1,0) 

W=b5:W 

E=b5:E 

N=b7:S(0,l) (1,0) 

U(0,1)  (l,0)=b2:D 
EN 

BC  b7 

D=b8:E(l,l) (0,0) 

S(0,1) (l,0)=b6;N 
W=b3 : E 
E=b3 :W 

N(0,1) (l,0)=b6:S 
U=b8:W(l,l) (0,0) 

EN 

BC  b8 

D(0,1) (l,0)=b5:D 
S(1,0) (0,l)=b4:S 
W(l,l) (0,0)=b7;U 
E(l,l) (0,0)=b7:D 
N(1,0) (0,l)=b4:N 
U(0,1) (l,0)=b5:U 


porb40.uif 


porg60h.uif 


o_porg60h 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
porg60h 

weights  one  half 
subcycling  lOx 


CHLAB3 

CHLAB4 


which  will  appear  at  the  start 
of  the  NOUT  channel  output 


' CHLAB5 

A 

**Label 

available  to  programmer 

1.1 

/ 

'NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

'NIN 

I 

**Current 

input  channel 

1.2 

/ 

'NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

'NONLIN 

I 

** Channel 

for  input-output 

1.2 

/ 

'NOUT 

I 

** Current 

output  channel 

1.2 

/ 

'NPRINT 

I 

** Channel 

for  printed  output 

1.2 

/ 

'NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

'NREC 

I 

** Cur rent 

record  number 

1.2 

/ 

'NRUN 

I 

**Maximum 

number  of  steps 

1.2 

/ 

'NADUMP 

lA 

**Codes 

for  array  dumps 

1.9 

/ 

'NPDUMP 

lA 

**Codes 

for  dumping  points 

1.9 

/ 

'NVDUMP 

lA 

**Codes 

for  dumping  arrays 

1.9 

/ 

'NLCHED 

L 

** .TRUE, 

if  class  0  report-head  required 

1.9 

/ 

'NLHEAD 

LA 

** .TRUE. 

if  class  1-9  report-heads  required 

1.9 

/ 

'NLOMTl 

LA 

**Class 

1 

subprogram  selector 

1.9 

/ 

'NLOMT2 

LA 

**Class 

2 

subprogram  selector 

1.9 

/ 

'NLOMT3 

LA 

**Class 

3 

subprogram  selector 

1.9 

/ 

'NLREPT 

L 

**.TRUE. 

if  report  required 

1.9 

/ 

'  NLGRAF 

L 

**di splay 

device  geometry 

7.3 

F/ 

SF  courant  number 
&SF  DTMUL=1.0, 

NRUN=400, 

CBLGLB='bla' ,XYZGLB=0.  ,  0.0005,0.  , 

MALG  ( 19  )  =1 , MALG  ( 20 )  =1 , 

NLDUMP=F, 

NXPTDD=5/ 

PS  pts  pclesO  alia  timesl 
FS  pclesO  pplotO 
FL  pplotO 

&FL  CFLNAM='y(x) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=-0 . 001 , FXMAX=0 . 001 , FMIN=-0 . 001 , FMAX=0 . 001/ 
PS  pts  pclesl  alia  timesl 
FS  pclesl  pplotl 
FL  pplotl 

&FL  CFLNAM=  'x{z)',LFSKIP=l,LFCOLO=3,  FXMIN=  0.0,  FXMAX=  0.002,  FMIN=  -0.001,  FMAX=  0.001/ 

DS  alia  alldo 

DO  alldo  bias  bibs  blcs  b4s  b3s  b2s  b7s  b6s  b5s 
SU  bias  bla 
SU  bibs  bib 
SU  blcs  blc 


SU  b4s  b4 
SU  b3s  b3 
SU  b2s  b2 
SU  bVs  b7 
SU  b6s  b6 
SU  b5s  b5 

TS  timesl  online  0  1000  1000 

BG  part2  polar_to_rectangular_transition  -4  -4  -6 

&POLRCT  RADCUR=0.001,RADSTR=0.0005,THEMIN=30. ,THEMAX=90. ,AXMIN=0. , AXMAX=0 . 002/ 
BG  parti  parallelepiped  -4  -4  -6 

&PIPED  RSID3  =  0 . 002 , RSID1=0 . 0005 , RSID2  =  0 . 0005 , THET1=90 . , THET2  =  90 .  , 

RPHIl=-90 . ,RPHI2=-30. / 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1./ 

BP  uniform2  uniform 


tUNIFRM  / 

BL  bla  parti  uniforml 
BL  bib  sameas  bla 
BL  blc  sameas  bla 
BL  b2  part2  uniforml 
BL  b3  sameas  b2 
BL  b4  sameas  b2 
BL  b5  sameas  b2 
EL  b6  sameas  b2 
BL  b7  sameas  b2 

SP  electrons  below  1  -1  0  0  8  10 
tPCLE  NPBGEO=l,CBLOCK='blb' , 
UCPOS=l. ,0.559,0.5,  -1., 
VEL=l.E+8, 0. ,0. ,NCVEL=1/ 

SP  eb  below  1  -1  0  0  8  10 
&PCLE  CBLOCK= ' b5 ' , 


porg60h.uif 

MALG(3) =10, 

UCPOS=0.5,1.0,0.5,  -1., 

VEL=l.E+8, 0. , 0 . ,NCVEL=1/ 

PO  bfield 

&SF  NINIT=1,BUNI=0.0,0.0,2.16/ 

PP  pcond  perf ect_conductor 
PA  pcond  saitieas  pcond 
BC  b2 

W(1,0) (0,1)=  blaiW 

S=b3:N 

N=b7 : S 

E=pcond 

U=b2 : D 

D=b2:U 

EN 

BC  b3 

W(1,0) (0,l)=bla:N 

S=b4:N 

N=b2 : S 

E=pcond 

U=b3:D 

D=b3:U 

EN 

BC  b4 

W(l, 0) (0,l)=blc:W 

S=b5:N 

N=b3 : S 

E=pcond 

U=b4:D 

D=b4:U 

EN 

BC  b5 

W{1,0) (0,l)=blc:N 

S=b6:N 

N=b4 : S 

E=pcond 

U=b5 : D 

D=bB:U 

EN 

BC  b6 

W(1,0) (0,l)=blb:W 

S=b7:N 

N=b5 : S 

E=pcond 

U=b6:D 

D=b6:U 

EN 

BC  b7 

W(1,0) (0,l)=blb:N 

S=b2 :N 

N=b6 : S 

E=pcond 

U=b7 : D 

D=b7:U 

EN 

BC  bla 

S=blb:E(l,0) (0,1) 

E=blc:S(l,0) (0,1) 

W=b2:W(l,0) (0,1) 

N=b3 :W(1, 0) (0,1) 

D(1,0) (0,l)=bla:U(l,0) (0,1) 

U(1,0) (0, 1) =bla:D(l, 0) (0,1) 

EN 

BC  blc 

S(1,0)  (0,l)=bla:E 
E=blb:S(l,0)  (0,1) 

W=b4:W(l,0) (0,1) 

N=b5:W(l,0) (0,1) 

D(1,0) (0,l)=blc:U(l,0) (0,1) 

U(1,0)  (0,l)=blc:D(l,0)  (0,1) 

EN 

BC  bib 

S(1,0)  (0,l)=blc:E 
E(1,0)  (0,l)=bla:S 
W=b6:W(l,0) (0,1) 

N=b7:W(l,0) (0,1) 

D(1,0) (0,l)=blb:U(l,0) (0,1) 

U(1,0) (0,l)=blb:D(l,0) (0,1) 


5/09/13:. 

8:11:27 

♦c  ,  , 


EN 

OR  bla  bib  blc  b4  b3  b2  b7  b6  b5 


porg60h.uif 


porgVO.uif 


o_porg70 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
porg70 

general  geometry 
periodic  case 

CHLAB3  which  will  appear  at  the  start 
CHLAB4  of  the  NOUT  channel  output 


'  CHLAB5 

A 

**Label  available  to  programmer 

1.1 

/ 

'NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

'NIN 

I 

**Current 

input  channel 

1.2 

/ 

'NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

'NONLIN 

I 

** Channel 

for  input-output 

1.2 

/ 

'NOUT 

I 

**Current 

output  channel 

1.2 

/ 

'  NPRINT 

I 

** Channel 

for  printed  output 

1.2 

/ 

'NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

'NREC 

I 

**Current 

record  number 

1.2 

/ 

'NRUN 

I 

*  * Maximum 

number  of  steps 

1.2 

/ 

'NADUMP 

lA 

**Codes  for  array  dumps 

1.9 

/ 

’  NPDUMP 

lA 

**Codes  for  dumping  points 

1.9 

/ 

'NVDUMP 

lA 

**Codes  for  dumping  arrays 

1.9 

/ 

'  NLCHED 

L 

*  * . TRUE .  , 

if  class  0  report-head  required 

1.9 

/ 

'  NLHEAD 

LA 

*  * . TRUE . 

if  class  1-9  report-heads  required 

1.9 

/ 

'NLOMTl 

LA 

**Class  1 

subprogram  selector 

1.9 

/ 

'NLOMT2 

LA 

**Class  2 

subprogram  selector 

1.9 

/ 

'  NL0MT3 

LA 

**Class  3 

subprogram  selector 

1.9 

/ 

'  NLREPT 

L 

**.TRUE. 

if  report  required 

1.9 

/ 

'NLGRAF 

L 

** display 

device  geometry 

7.3 

F/ 

SF  courant_nuinber 
&SF  0TMUL=1.O, 

NRUN=400, 

NXPTDD=5 , 

MALG{19)=1,MALG(20)=1, 

NLDUMP=F , 

NXPTDD=5/ 

PS  pts  pclesO  alia  timesl 
FS  pclesO  pplotO 
FL  pplotO 

&FL  CFLNAM='y(x) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=0 . , FXMAX=0 . 002 , FMIN=0 . ,FMAX=0.002/ 
PS  pts  pclesl  alia  timesl 
FS  pclesl  pplotl 
FL  pplotl 

&FL  CFLNAM='x(z) ' , LFSKIP=1 , LFCOLO=3 , FXMIN=0 . 0 , FXMAX=0 . 005 , FMIN=0 . ,FMAX=0.002/ 
DS  alia  alldo 
DO  alldo  bis 
SU  bis  bl 

TS  timesl  online  0  1000  1000 
BG  typel  parallelepiped  -10  -10  -10 

&PIPED  RSID1=2 .236068E-3 ,RSID2=2 .4494897E-3,RSID3=l.E-3, 

THET1=26 . 56505, THET2=35 . 26439 , RPHI1=0 . ,RPHI2=45/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1 . , RMUR= 1 . / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 
SP  electrons  below  1  -1  0  0  8  10 
&PCLE  NPBGEO=l,CBLOCK='bl' , 

UCPOS=0.5,0.5,0.5,  -1., 

VEL=l.E+8,0. ,0. ,NCVEL=1/ 

SP  eb  below  1  -1  0  0  8  10 
&PCLE  CBLOCK= ' bl ' , 

UCPOS=0.5,0.5,0.5,  -1., 

VEL=l.E+8,0. ,1.5E+7,NCVEL=1/ 

PO  bfield 

&SF  NINIT=1,BUNI=0.0, 0.0,2.16/ 

BC  bl 
E=bl:W 
D=bl : U 
U=bl : D 
S=bl:N 
N=bl : S 
W=bl : E 
EN 

OR  bl 


mm3 

f08;ll:29. 


o_sbcl6z 

'NLRES  L  False  for  NEWRUN.True  for  RESET' 
sbcl6z 

TM  mode  in  cube  as  be 


F 


sbcl6z.uif 


CHLAB5 

A 

** Label 

available  to  programmer 

1.1 

/ 

NDIARY 

I 

** Channel 

for  diary 

1.2 

/ 

NIN 

I 

**Current 

input  channel 

1.2 

/ 

NLEDGE 

I 

** Channel 

for  restart  records 

1.2 

/ 

NONLIN 

I 

** Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

** Cur rent 

output  channel 

1.2 

/ 

NPRINT 

I 

** Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

** Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

♦♦Current 

record  number 

1.2 

/ 

NRUN 

I 

♦ ♦Maximum 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

♦♦Codes 

for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

♦♦Codes 

for  dumping  points 

1.9 

/ 

NVDUMP 

lA 

♦ ♦Codes 

for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

♦♦ .TRUE. 

.  if  class  0  report-head  required 

1.9 

/ 

NLHEAD 

LA 

♦ ♦ . TRUE . 

if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

♦♦Class 

1 

subprogram  selector 

1.9 

/ 

NLOMT2 

LA 

♦♦Class 

2 

subprogram  selector 

1.9 

/ 

NLOMT3 

LA 

♦♦Class 

3 

subprogram  selector 

1.9 

/ 

NLREPT 

L 

♦♦ .TRUE. 

if  report  required 

1.9 

/ 

NLGRAF 

L 

♦♦display 

device  geometry 

7.3 

F/ 

SF  courant_nuinber 
&SF  DTMUL=1.0, 

NINIT=5, 

NRtIN=200, 

NXPTDD=5, 

AMODE=0.01,0.01,0.01 
CM0DE=1 . 

NM0DE=1, 1, 1/ 

PS  tsd  ptflds  points  times 
FS  ptflds  El  E2  E3  HI  H2  H3 

DS  points  ptl  pt2  pt3  pt4  pt5  pt6  ptlt  pt2t  pt3t  pt4t  pt5t  pt6t 
SUptl  bl  1  2  3  0.5  0.5  0.  0.5  0.5  0. 

SU  pt2  b2  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SUpt3  b3  1  2  3  0.5  0.5  0.4  0.5  0.5  0.4 

SU  pt4  b4  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  pt5  b5  1  2  3  0.5  0.5  0.8  0.5  0.5  0.8 

SU  pt6  b6  1  2  3  0.5  0.5  1.0  0.5  0.5  1.0 

SU  ptlt  bl  1  2  3  0.0  0.5  0.  0.5  0.5  0. 

SU  pt2t  b2  1  2  3  0.0  0.5  0.2  0.5  0.5  0.2 

SU  pt3t  b3  1  2  3  0.0  0.5  0.4  0.5  0.5  0.4 

SU  pt4t  b4  1  2  3  0.0  0.5  0.6  0.5  0.5  0.6 

SU  pt5t  b5  1  2  3  0.0  0.5  0.8  0.5  0.5  0.8 

SU  pt6t  b6  1  2  3  0.0  0.5  1.0  0.5  0.5  1.0 

TS  times  online 

PS  tsd  avfld  face  times2 

FS  avfld  intD.dS  intB.dS  intj .dS 

DS  face  onbl  onb2  onb3  onb4 

SU  onbl  bl  1231.  0.  0.  1.  1.  1. 

TS  times2  online  0.  10.  0.002  0.  0.  9 
BG  typel  regular_cubic_lattice  -6  -8  -10 
&CUBREG  RXMAX=0 . 01 , RYMAX=0 . 01 , RZMAX=0 .01/ 

BP  uniforml  uniform 
ScUNIFRM  EPSR=1.  ,RMUR=1.  / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 
PP  pcond  perfect_conductor 
PP  reswal  resistive_wall 
ScBCS  SURFZ=217.51,STHETA=0.5  / 

SG  boundary  signalcpt 
SC  signalcpt 

&SIGNAL  FREQUENCY=2.598E+10,DURATION=100.  ,  SIGNAL_FORM=  '  CONSTANT '  / 
PP  surfld  surface_f ield 

ScBCS  DAPLYA  { 1 )  =0 . 0 ,  DAPLYA  (2  )  =0 . 0  ,  DAPLYA  ( 3  )  =1 . 0  , 

EAMP=1000. ,NGLOBC=l, SFORM=' boundary' / 

PA  pcond  sameas  pcond 
PA  reswal  sameas  reswal 
PA  surfld  sameas  surfld 
BC  bl 
E=pcond 
D=surf Id 


sbcl6z.uif 


U=reswal 

S=pcond 

N=pcond 

W=pcond 


EN 

OR  bl 


tp06.uif 

o_tp06 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
tp06 

TElll  mode 
periodic  case 

cylinder  with  parallelogram  cross-section 


'  CHLAB5 

A 

■“Label 

available  to  programmer 

1.1 

'  / 

'NDIARY 

I 

*■*  Channel 

for  diary 

1.2 

/ 

'NIN 

I 

* ‘Current 

input  channel 

1.2 

/ 

NLEDGE 

I 

■“Channel 

for  restart  records 

1.2 

/ 

NONLIN 

I 

*  ■‘Channel 

for  input-output 

1.2 

/ 

NOUT 

I 

“Current 

output  channel 

1.2 

/ 

NPRINT 

I 

“Channel 

for  printed  output 

1.2 

/ 

NPUNCH 

I 

“Channel 

for  card  output  (or  equivalent) 

1.2 

/ 

NREC 

I 

“Current 

record  number 

1.2 

/ 

NRUN 

I 

“Maximiun 

number  of  steps 

1.2 

/ 

NADUMP 

lA 

“Codes 

for  array  dumps 

1.9 

/ 

NPDUMP 

lA 

“Codes 

for  dumping  points 

1.9 

/ 

NVDUMP 

lA 

“Codes 

for  dumping  arrays 

1.9 

/ 

NLCHED 

L 

“.TRUE. 

if  class  0  report-head  required 

1.9 

/ 

NLHEAD 

LA 

“.TRUE, 

,  if  class  1-9  report-heads  required 

1.9 

/ 

NLOMTl 

LA 

“Class 

1 

subprogram  selector 

1.9 

/ 

NLOMT2 

LA 

“Class 

2 

subprogram  selector 

1.9 

/ 

NLOMT3 

LA 

“Class 

3 

subprogram  selector 

1.9 

/ 

NLREPT 

L 

‘ ‘ . TRUE , 

.  if  report  required 

1.9 

/ 

NLGRAF 

L 

“display 

device  geometry 

7.3 

F/ 

SF  courant_number 
&SF  DTMUL=1.0, 

NINIT=4 , 

NRUN=200, 

NXPTDD=5, 

AM0DE=1. , 1 .  ,  1 . 

CM0DE=1. 

NMODE=:l,  1, 1/ 

PS  tsd  ptflds  points  times 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  bitbl 

SU  bitbl  bl  1  2  3  0.5  0.5  0.5  0.5  0.5  0.5 

TS  times  online 

PS  tsd  avfld  face  times2 

FS  avfld  intD.dS  intB.dS  intj.dS 

DS  face  onbl 

SU  onbl  bl  12  31.  0.  0.  1.  1.  1. 

TS  times2  online  0.  10.  0.2  0.  0.  9 
BG  typel  quadrilateral_cylinder  -10  -10  -10 
&QUADRI  XQUAD1=2 . , YQUAD1=2 . , XQUAD2=4 . , YQUAD2=2 . , 
RXMAX=2  .  ,  RYMAX=0  .  ,  RZMAX=2  .  / 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1./ 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 

BC  bl 

E=bl:W 

D=bl:U 

U=bl : D 

S=bl:N 

N=bl : S 

W=bl ; E 

EN 

OR  bl 


uneSl.uif 

o_une51 

'NLRES  L  False  for  NEWRUN,True  for  RESET'  F 
uneSl 

TElll  mode 

All  orientations  of  cubes 


CHLAB3 
CHLAB4 
' CHLAB5 

which  will  appear  at  the  start 
of  the  NOUT  channel  output 

A  **Label  available  to  programmer 

1.1 

/ 

'NDIARY 

I 

**Channel  for  diary 

1.2 

/ 

'NIN 

I 

♦♦Current  input  channel 

1.2 

/ 

'NLEDGE 

I 

♦♦Channel  for  restart  records 

1.2 

/ 

'NONLIN 

I 

♦♦Channel  for  input-output 

1.2 

/ 

'NOUT 

I 

♦♦Current  output  channel 

1.2 

/ 

'NPRINT 

I 

♦♦Channel  for  printed  output 

1.2 

/ 

'NPUNCH 

I 

♦♦Channel  for  card  output  (or  equivalent) 

1.2 

/ 

'NREC 

I 

♦♦Current  record  number 

1.2 

/ 

'NRUN 

I 

♦♦Maximum  number  of  steps 

1.2 

/ 

'NADUMP 

lA 

♦♦Codes  for  array  dumps 

1.9 

/ 

'NPDUMP 

lA 

♦♦Codes  for  dumping  points 

1.9 

/ 

'NVDUMP 

lA 

♦♦Codes  for  dumping  arrays 

1.9 

/ 

' NLCHED 

L 

♦♦.TRUE,  if  class  0  report-head  required 

1.9 

/ 

'NLHEAD 

LA 

♦♦.TRUE,  if  class  1-9  report-heads  required 

1.9 

/ 

'NLOMTl 

LA 

♦♦Class  1  subprogram  selector 

1.9 

/ 

' NLOMT2 

LA 

♦♦Class  2  subprogram  selector 

1.9 

/ 

' NLOMT3 

LA 

♦♦Class  3  subprogram  selector 

1.9 

/ 

' NLREPT 

L 

♦♦.TRUE,  if  report  required 

1.9 

/ 

'  NLGRAF 

L 

♦♦display  device  geometry 

7.3 

F/ 

SF  courant_number 
ScSF  DTMUL=1.0, 

NINIT=4, 

NRUN=100, 

NXPTDD=5, 

AM0DE=1. ,1. ,1. 

CM0DE=1. 

NMODE=l, 1, 1/ 

PS  tsd  ptflds  points  times 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  bitbl 

SU  bitbl  bll231.  1.  1.  1.  1.  1. 
TS  times  online 
PS  tsd  avfld  face  times2 
FS  avfld  intD.dS  intB.dS  intj.dS 
DS  face  onbl  onb2  onb3  onb4 


SU 

onbl 

bl 

1 

2 

3 

1. 

0. 

0. 

1. 

1. 

1 

SU 

onb2 

b2 

1 

2 

3 

1. 

0. 

0. 

1. 

1. 

1 

SU 

onb3 

b3 

1 

2 

3 

0. 

0. 

0. 

0. 

1. 

1 

SU 

onb4 

b4 

1 

2 

3 

0. 

0. 

0. 

0. 

1. 

1 

PS  tsd  energy  volume  times 

FS  energy  intE.D/2dV  intB.H/2dV  intj.EdV 

DS  volume  alibi  allb2  allb3  allb4 

SU  alibi  bl  1  2  3 

SU  allb2  b2  1  2  3 

SU  allb3  b3  1  2  3 

SU  allb4  b4  1  2  3 

TS  times2  online  0.  10.  0.2  0.  0.  9 
BG  typel  regular_cubic_lattice  -3  -4  -5 
&CUBREG  RXMAX=1.0,RYMAX=1.0,RZMAX=1.0/ 

BG  type2  regular_cubic_lattice  -3  -4  -5 
&CUBREG  RXMAX=1.0,RYMAX=1.0,RZMAX=1.0/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1. ,RMUR=1. / 

BP  uniform2  uniform 
&UNIFRM  / 

BL  bl  typel  uniforml 
BL  b2  type2  uniforml 
BL  b3  sameas  bl 
BL  b4  scimeas  bl 
BL  b5  sameas  bl 
BL  b6  sameas  bl 
BL  b7  sameas  bl 
BL  b8  sameas  bl 
PP  pcond  perfect_conductor 
PP  reswal  resistive_wall 
&BCS  SURFZ=377. ,STHETA=0.5  / 

PP  aplfld  applied_field 
&BCS  DAPLYA(1)=115.5712,DAPLYA(2)=0. , 

PA  pcond  sameas  pcond 
PA  reswal  sameas  reswal 


DAPLYA(3)=0.  / 


une51.uif 


PA  aplfld  sameas  aplfld 
BC  bl 

D=b5:N(l,0) (0,1) 

S=b4:D 

W=b2:N(l,0) (0,1) 
E=b2;S(l,0) (0,1) 

N=b4 :U 

U=b5:S(l,0) (0,1) 

EN 

BC  b2 

D=b6:U(0,l) (1,0) 

S(1,0) (0,l)=bl:E 
W=b3 :S(0, 1) (1,0) 
E=b3:N(0,l) (1,0) 

N(1,0)  (0,l)=bl:W 
U=b6 :D(0,1)(1,0) 

EN 

BC  b3 

D=b4:W{0,l) (1,0) 

S(0,1) (l,0)=b2:W 
W=b7 : E 
E=b7 : W 

N(0,1) (l,0)=b2:E 
U=b4:E(0,l) (1,0) 

EN 

BC  b4 
D=bl : S 

S=b8:S(l, 0) (0,1) 

W(0,1)  (l,0)=b3:D 
E(0,1) (l,0)=b3:U 
N=b8 :N(1, 0) (0, 1) 

U=bl:N 

EN 

BC  b5 

D=b8:D(0, 1) (1,0) 

S(1,0)  (0,  l)=bl:U 

W=b6;W 

E=b6:E 

N(1,0) (0,l)=bl:D 
U=b8:U{0,l) (1,0) 

EN 

BC  b6 

D(0,1) (l,0)=b2;U 
S=b7:N(0, 1) (1,0) 

W=b5:W 

E=b5:E 

N=b7:S(0, 1) (1,0) 

U(0,1) (l,0)=b2:D 
EN 

BC  b7 

D=b8:E(l, 1) (0,0) 

S(0,1) (l,0)=b6:N 
W=b3 : E 
E=b3 :W 

N(0,1) (1, 0)=b6:S 
U=b8:W(l, 1) (0,0) 

EN 

BC  b8 

D(0,1) (l,0)=b5:D 
S(1,0) (0,l)=b4:S 
W(l,l) (0,0)=b7:U 
E(l,l) (0,0)=b7:D 
N(1,0) (0, l)=b4:N 
U(0,1) (l,0)=b5:U 
EN 

OR  bl  b2  b3  b4  b5  b6  b7  b8 
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J  W  Eastwood,  W  Arter,  N  J  Brealey 
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Abstract 

This  Note  contains  provides  an  introduction  to  the  user  input  files 
(.uif),  and  running  the  preprocessor  PEGGIE  the  simulation  code  PIC3D 
postprocessors  TSDMERGE,  MPICTIM  and  DISPAN  and  the  GHOST  graph¬ 
ical  system. 


1  Introduction 

Performing  a  simulation  using  the  3DPIC  software  suite  is  a  three  stage  process: 

1.  Prepare  a  user  input  (*.uif)  file  and  run  the  PEGGIE  preprocessor  to 
generate  the  input  data  (*.dat)  file  for  the  main  simulation  program 

PIC3D. 

2.  Run  the  main  simulation  program  PIC3D  to  generate  the  output  files 

*.res  restart  file  for  extending  a  PIC3D  simulation 
* .  out  text  output  (mainly  diagnostic)  data 
*.tsd  timeseries  input  dataset  for  MPICTIM 

*  .pnt  ASCII  version  of  *  .tsd  file 
*.grd  GHOST  grid  file  of  initial  data 

*  .grl  GHOST  grid  file  of  line  graph  snapshots 
* .  lin  ASCII  version  of  * .  grl  file 

*.gst  GHOST  grid  file  of  scatter  plot  snapshots 
* .  pt  s  ASCII  version  of  * .  gst  file 

If  PIC3D  is  run  on  a  several  processors,  then  each  process  generates  an 
output  file.  The  GHOST  grid  file  outputs  can  be  combined  by  superposing 
them  using  the  interactive  GHOST  postprocessor  xghost.  The  timeseries 
(*  .tsd)  files  from  each  process  are  combined  using  the  postprocessor  TS¬ 
DMERGE  before  exporting  the  files  back  to  the  workstation  on  which 
MPICTIM  is  run. 

3.  Run  the  postprocessors  (MPICTIM,  DISPAIM,  GHOST)  to  analyse  the  out¬ 
put  files. 

The  following  Sections  give  examples  of  running  the  various  components  of 
the  3DPIC  software  suite. 
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2  Preprocessing  using  PEGGIE 

To  run  the  preprocessor  PEGGIE  and  generate  the  data  file  for  the  main  simu¬ 
lation  run  issue  the  peggie  command  with  the  name  of  the  user  input  file  as 
the  argument.  For  example, 


7o  peggie  ml407.uif 


This  generates  the  output  file  ml407 .  dat  for  use  by  the  main  simulation  code 
P1C3D  and  optionally  a  graphical  output  file  containing  a  wire  grid  plot  of  the 
element  net  generated  by  PEGGIE  with  colour  coding  of  each  block.  The  present 
Version  of  PEGGIE  generates  a  large  number  of  diagnostic  warning  messages, 
which  can  be  ignored  (but  error  messages  should  not  be  ignored  if  they  appear!), 
and  then  goes  on  to  output  prompts  for  the  interactive  choice  of  parameters  for 


grid  display: 

VPNX 

-l.OOOOE+00 

VPNY 

-3.0000E-01 

VPNZ 

7.0000E-01 

VRCX 

O.OOOOE+00 

VRCY 

O.OOOOE+00 

VRCZ 

O.OOOOE+00 

VUPX 

O.OOOOE+00 

VUPY 

l.OOOOE+00 

VUPZ 

O.OOOOE+00 

xmin  map  = 

-1.2500E-01 

-400 


xmin  map  CHANGED  TO  -4.0000E+02 
xmax  map  =  1.2500E-01 

400 

xmeix  map  CHANGED  TO  4.0000E+02 
ymin  map  =  -1.2500E-01 
-400 

ymin  map  CHANGED  TO  -4.0000E+02 
ymeix  map  =  1.2500E-01 

400 

ymax  map  CHANGED  TO  4.0000E+02 


plotype 
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Figure  1:  The  wire  grid  plot  of  block  edges  generated  by  the  inl407  .uif  example 
run  of  PEGGIE. 

3 

plotype  CHANGED  TO  3 

another  projection?  y/n 
n 

In  the  above  dialogue,  the  user  reply  /  to  the  displayed  value  leaves  that  value 
unchanged.  The  first  three  items  VPNX,  VPNY  and  VPNZ  specify  the  vector  giving 
the  viewing  position,  the  next  three  (VRCX,  VRCY  and  VRCZ)  define  the  reference 
origin  point  on  the  element  net,  and  the  third  triplet  define  the  ‘up’  direction 
for  the  plot. The  minimum  and  maximum  map  values  define  the  extent  of  the 
mapping  to  the  viewing  plane  to  be  displayed,  plotype  is  1  for  all  element 
edges  to  be  plotted,  2  for  element  edges  on  block  faces  to  be  plotted,  and  3  for 
only  block  edges  to  be  plotted.  The  result  for  this  example  is  shown  in  Fig  1 

2.1  Input  file  nil407.uif 

In  this  Subsection,  a  step-by-step  discussion  of  a  sample  .uif  file  is  given  to 
illustrate  the  method  of  building  up  geometry,  etc  using  parts.  A  full  definition 
of  the  input  formats  used  in  the  .uif  files  is  given  in  the  Note  [7],  and  that 
document  should  be  referred  to  when  reading  the  following  example. 

The  example  input  file  used  is  ml407  .uif  which  sets  up  a  four  cavity  MILO, 
using  a  cylindrical  block  for  the  diode  region  and  a  set  of  120  degree  angle 
annular  segments  for  the  drift  space  and  cavities.  This  MILO  example,  and  full 
.uif  files  for  a  number  of  other  test  cases  is  presented  in  Reference  [6]. 

The  initial  part  of  the  ml407 .  uif  file  is  concerned  with  labelling;  the  only 
important  label  is  the  reference  labelling  string  ml407  which  propagates  through 
all  the  files  generated  by  3DPIC  and  should  be  unique  to  each  simulation  ex¬ 
periment  to  prevent  filename  clashes  with  other  runs. 

The  line 

'NLGRAF  L  **display  device  geometry  7.3  '  T/ 

is  used  to  select  the  display  of  the  device  geometry.  If  the  T/  is  replaced  by  F/, 
then  the  device  geometry  is  not  plotted. 
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set  free  parameter  The  tag  SF  indicates  that  the  SF  namelist  follows  with 
the  various  free  parameters  to  be  set.  In  this  example,  the  Courant  Number 
is  set  to  0.99  (DTMUL),  the  run  length  is  2000  steps  and  diagnostic  output  is 
suppressed  (NXPTDD).  NTPRES  is  the  number  of  processors  on  the  target  machine. 
The  \  terminates  the  namelist. 

SF  courant  number 
&SF  DTMUL=0.99, 

NRUN=2000 , 

NTPRES=1, 

NXPTDD=5/ 

scatter  plots  The  next  sequence  specifies  the  scatter  plots.  The  fist  line  (tag 
PS)  defines  the  plot  set 

PS  gst  poles  alia  times 

This  says  that  the  plot  set  is  a  grid  file  scatter  plot  type  (-gst)  for  the  field  set 
pcles  over  domain  set  alia  for  the  time  set  times. 

The  following  extract  defines  the  field  set  pcles: 

FS  pcles  pplotl  pplot2 
FL  pplotl 

&FL  CFLNAM= 'r (z) ' , LFSKIP=10 , LFC0L0=3 . 

FXMIN=-0 . 2 , FXMAX=0 . 05 , FMIN=0 . , FMAX=0 . 150/ 

FL  pplot2 

&FL  CFLNAM='theta(z) ',LFSKIP=10,LFC0L0=4, 

FXMIN=-0 . 2 ,FXMAX=0 . 05 ,FMIN=-180 . ,FMAX=180 . / 

This  field  set  (tag  FS)  contains  fields  pplotl  and  pplot2.  Parameters  for  the 
field  pplotl  are  given  by  the  namelist  following  the  FL  tag  for  that  field;  in 
this  instance  it  is  a  r  —  z  scatter  plot,  plotting  every  tenth  particle  in  GHOST 
colour  3  (green)  with  the  abscissa  variable  (z)  over  range  -0.2  to  0.05  and  the 
ordinate  variable  over  range  0  to  0.15.  Similarly  pplot2  is  a  0  —  z  scatter  plot, 
this  time  using  blue  dots  for  the  particles. 

The  domain  set  alia  is  defined  by  the  DS  line;  in  this  case,  there  is  one 
domain  named  alldo,  comprising  sub-domains  bis  ■  •  •  b4Es.  The  lines  tagged 
by  SU  link  the  subdomains  to  whole  blocks  in  this  instance. 

DS  alia  alldo 

DO  alldo  bis  b2s  b3s  b4s  ca2s  ca3s  ca4s  b2cs  b3cs  b4cs  ca2cs  + 

ca3cs  ca4cs  b2bs  b3bs  b4bs  ca2bs  ca3bs  ca4bs  b2as  b3as  b4as  + 

ca2as  ca3as  ca4as  b2Es  b3Es  b4Es 

SU  bis  bl 

SU  b2s  b2 

SU  b3s  b3 

SU  b4s  b4 

SU  ca2s  ca2 

SU  ca3s  ca3 
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The  time  set  times  is  defined  later  in  the  .uif  file.  Since  this  is  the  first 
plot  set  defined,  it  will  lead  to  the  output  file  ml3407-01  .gst  being  generated 
when  PIC3D  is  executed  on  a  single  processor  workstation.  On  a  multiprocessor 
machine,  the  process  number  is  also  appended  to  the  root  of  the  file  name  (eg 
ml3407-01-0002.gst  for  process  2  output  and  so  forth). 


point  field  plots  The  second  plot  set,  whose  output  will  go  to  the  output 
file  ml3407-02.tsd  is  a  time  series  data  file  format,  and  will  contain  point  field 
values  defined  by  the  field  set  ptflds  for  the  domain  set  points  sampled  for 
time  set  times2. 

PS  tsd  ptflds  points  times2 
FS  ptflds  El  E2  E3  HI  H2  H3 
DS  points  ptl  pt2  pt3  pt4  pt5 
SUptl  b2E  1  2  3  0.5  0.5  0.0  0.5  0.5  0.0 

SU  pt2  b2a  1  2  3  0.5  0.5  0.6  0.5  0.5  0.6 

SU  pt3  b2c  1  2  3  0.5  0.5  0.2  0.5  0.5  0.2 

SUpt4b2  1230. 5  0.5  0.0  0.5  0.5  0.0 

SU  pt5  b2  123  0.5  0.5  0.6  0.5  0.5  0.6 

The  sub-domain  definitions  are  a  little  more  involved  than  for  the  scatter 
plot  case,  with  three  triplets  of  numbers  after  the  block  name.  The  first  gives 
the  direction  for  evaluating  components  -  in  this  case  the  block  ,x^)  are 

the  (1,  2,  3)  components,  respectively.  The  next  two  triplets  the  ‘origin’  point 
and  the  last  is  the  ‘extreme’  point  which  define  the  diagnostic  volume  in  block 
coordinates  with  components  on  interval  [0,1].  For  instance,  pt5  is  at  point 
(0.5,  0.5,  0.6). 

The  next  section  of  the  .uif  file  defines  a  second  .tsd  file,  this  time  for 
points  in  the  cavity  blocks. 

PS  tsd  ptflds  cavs  times2 
DS  cavs  cav2  cav3  cav4  cav5 

SU  cav2  ca2a  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav3  ca2b  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav4  ca2c  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 

SU  cav5  ca2  1  2  3  0.8  0.5  0.5  0.8  0.5  0.5 
TS  times  online 

TS  times2  online  0.  50.  0.006  0.  0.  9 

The  last  two  lines  of  this  fragment  define  the  time  sets  times  and  times2.  The 
time  set  times  specifies  that  data  be  collected  at  default  ‘sensible’  times.  The 
times2  time  set  specifies  that  data  be  collected  between  times  0  and  50  at 
intervals  .006,  where  the  9  indicates  that  time  units  are  nanoseconds. 
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line  graphs  The  section  defining  the  line  graph  follows  the  same  pattern  as 
the  other  diagnostics,  with  definitions  of  plot  set,  field  set  and  domain  set.  The 
domain  outer  is  a  curve,  and  the  tag  CU  introduces  the  namelist  INHERI,  which 
defines  the  start  point  (0,  0,  0.8)  on  block  b2b  for  a  curve  which  propagates 
along  element  edges  in  the  2-direction  through  surface  patches  to  neighbouring 
blocks  until  it  closes  or  terminates  at  a  boundary  surface.  In  this  case,  the 
curve  is  a  closed  circular  curve  enclosing  the  cathode. 

PS  grl  Efield  outerline  times 
FS  Efield  El  E2  E3 
DS  outerline  outer 
CU  outer  inherit 
&INHERI  CBL0CK='b2b', 

0P0INT=0. ,0. ,0.8, 

NDIR(1)=2, 

NLXTEN=T, CUNIVS= 'all  7 

surface  integrals  The  final  plot  set  generates  a  .tsd  file  which  gives  radial 
fluxes  through  a  surface  in  block  b2. 

PS  tsd  avfld  face  times2 
FS  avfld  intD.dS  intB.dS  intj.dS 
DS  face  onb2 

SU  onb2  b2  1  2  3  0.4  0.  0.  0.4  1.  1. 

block  types  The  above  completes  the  specification  of  the  diagnostics.  The 
next  stage  is  to  define  the  geometry  and  physics  of  the  parts  which  make  up 
the  block  types  used  in  describing  a  device.  Tag  BG  flags  block  geometry  and 
BP  flags  block  physics. 

BG  centre  polar_with_regular_meshing  -8  -12  -10 
&P0LREG  RADINR=0. , RADDUT=0. 0375, THEMAX=360. ,AXMAX=0. 025/ 

BG  outer  polar _with_regular .meshing  -8  -4  -16 

&P0LREG  RADINR=0.0375,RAD0UT=0.075,THEMAX=120. ,AXMAX=0.04/ 

BG  cavity  polar_with_regular .meshing  -16  -4  -16 
&P0LREG  RADINR=0 . 075 , RAD0UT=0 . 150 , THEMAX=120 . , AXMAX=0 . 04/ 

BG  inout  polar.with.regular.meshing  -8  -4  -20 

&PDLREG  RADINR=0 . 0375 , RAD0UT=0 . 075 ,THEMAX=120 . , AXMAX=0 . 05/ 

BG  first  polar.with.regular.meshing  -8  -4  -4 

&P0LREG  RADINR=0.0375,RAD0UT=0.075,THEMAX=120. ,AXMAX=0.01/ 

BP  uniforml  uniform 
&UNIFRM  EPSR=1 . , RMUR=1 . / 

BP  uniform2  uniform 


All  the  BG  parts  defined  above  are  of  the  class  polar.with.regular.meshing. 
The  first  part  (centre)  has  a  suggested  mesh  of  8  x  12  x  10  elements,  and 
forms  a  complete  cylinder  {Omax  =  360)  with  outer  radius  0.0375  m,  axial 
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length  of  0.025  m.  Both  block  physics  parts  are  of  class  uniform  with  relative 
permitivitty  and  permeability  of  unity. 

blocks  Each  block  has  a  label,  a  geometry  part  and  a  physics  part.  For 
example,  block  b2  has  an  outer  block  geometry  and  a  unif  orml  block  physics. 
Block  b3  has  the  same  block  geometry  and  physics  and  block  b2  and  so  forth. 

BL  bl  centre  uniforml 
BL  b2  outer  uniforml 
BL  b3  sameas  b2 
BL  b4  sameas  b2 
BL  ca2  cavity  uniforml 
BL  ca3  sameas  ca2 
BL  ca4  sameas  ca2 
BL  b2a  inout  uniforml 


surface  patches  The  whole  of  the  exterior  surface  of  the  device  must  be  cov¬ 
ered  by  surface  patches,  and  each  of  these  patches  have  associated  patch  physics. 
The  patch  physics  tags  (PP)  and  associated  boundary  condition  namelists  be¬ 
low  introduce  perfect  conductors  (pcond),  a  377  Q  per  square  resistive  surface 
(reswal),  a  radial  electric  field  corresponding  to  an  applied  voltage  of  ampli¬ 
tude  0.5  MV  with  Er  radially  inwards,  and  a  polar  axis.  The  namelist  quantity 
SFORM  specifies  the  signal  waveform  part  -  in  this  case  boundary  (cf  applied  sig¬ 
nal  paragraph  below) .  The  PA  tags  then  associate  these  electromagnetic  patch 
physics  conditions  with  patch  names. 

PP  pcond  perf ect_conductor 
&BCS  / 

PP  reswal  resistive_wall 
&BCS  SURFZ=377 . , STHETA=1 . 0  / 

PP  aplfld  applied_f ield 

&BCS  DAPLYA(1)=-1. ,DAPLYA(2)=0. ,DAPLYA(3)=0. , 

DP0T=0 . 5E+6 , SF0RM= 'boundary '/ 

PP  eixis  polar_axis 
&BCS  / 

PA  pcond  sameas  pcond 
PA  reswal  sameas  reswal 
PA  aplfld  sameas  aplfld 
PA  axis  sameas  axis 

If  charged  particle  source  boundary  conditions  are  associated  with  a  patch, 
then  the  patch  definition  requires  both  particle  and  electromagnetic  patch 
physics.  The  following  species  definition  (tag  SP)  defines  species  electrons 
of  class  any  of  mass  charge  —  |  e  |  with  10^  particles  per  superparticle,  a 
nominal  8  particles  per  element  and  a  subcycling  of  1  electromagnetic  timesteps 
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per  particle  timestep.  The  PP  tag  defines  part  emitter  to  be  of  class  emitter 
where  the  class  emitter  namelist  PARBCS  allow  particle  boundary  condition 
parameters  to  be  set.  The  patch  definition  tag  PA  associates  emitter  parti¬ 
cle  patch  physics  and  pcond  electromagnetic  particle  boundary  conditions  with 
patch  part  pemcon. 

SP  electrons  ainy  1  -1  6  9  8  1 

PP  emitter  emitter 

&PARBCS  CSPECI= 'electrons ' , EFFE=1 . 0/ 

PA  pemcon  emitter  pcond 

applied  signals  The  signal  part  boundary  has  component  parts  ramp  and 
signalcpt;  ramp  applies  a  linear  ramp  up  from  0  to  the  signal  voltage  (DPOT) 
over  an  interval  of  3  nanoseconds,  and  signalcpt  hold  the  voltage  constant 
over  the  next  10,000  seconds. 

SG  boundary  ramp  signalcpt 

SC  ramp 

&SIGNAL 

FREQUENCY  =0.0, 

DURATION  =  0.3E-9, 

POWER  =  0.0,  1.0, 

SIGNAL_F0RM= 'LINEAR'/ 

SC  signalcpt 
&SIGNAL 

DURATION  =  10000 . , 

POWER  =  1.0,  1.0, 

SIGNAL_F0RM= 'CONSTANT ' / 

block  connections  The  final  task  is  to  glue  all  the  blocks  together  by  spec¬ 
ifying  the  block  connectivity  for  each  block.  There  is  some  redundancy  in  this 
specification,  but  this  is  checked  by  PEGGIE  in  its  consistency  checks  when 
building  the  patch  tables. 

BC  b2 

W(0,0) (l,0.375)=pemcon 
W(0,0.375)(l,l)=  bl:E  (0,  0) (0 . 333333, 1) 

N=b3 : S 

S=b4:N 

E=ca2:W 

D=b2c:U 

Up  =pcond 

EN 


BC  ca2 
W=b2:E 
N=ca3:S 
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S=ca4:N 

E=pcond 

D=pcond 

U=pcond 

EN 


Each  block  has  a  north,  south  east,  west,  up  and  down  face,  as  described  in 
[7].  For  full  face  connection  with  normal  orientation,  one  simply  has  to  specify 
which  face  of  which  target  block  the  current  block  is  being  connected.  Thus, 
for  cavity  block  ca2,  the  west  face  is  connected  to  the  east  face  of  block  b2,  the 
north  face  is  connected  to  the  south  of  block  ca3,  the  south  face  is  connected 
to  the  north  face  of  block  ca4,  and  the  east,  down  and  up  faces  are  connected 
to  perfect  conductors. 

For  part-face  connections,  the  local  block  coordinates  of  the  ‘O’  and  ‘X’ 
corners  of  the  connected  parts  of  the  face  also  need  specifying.  Thus,  for  the 
120  degree  annular  wedge  block  b2,  the  west  face  (ie  the  surface  at  RADINR)  is 
partly  covered  by  the  cathode  emitting  conductor  surface  pemcon  (for  from  0 
to  0.375)  and  partly  connected  (for  x^  from  0.375  to  1)  to  the  east  face  (between 
^  =  0  and  6  =  120  degrees)  of  block  bl. 

Finally,  the  order  in  which  the  block  connection  (tag  OR)  can  be  specified 
to  determine  which  block  meshing  density  takes  priority  if  inconsistencies  are 
detected. 

OR  bl  b2  b3  b4  ca2  ca3  ca4  b2c  b3c  b4c  ca2c  ca3c  ca4c  b2b  b3b 
b4b  ca2b  ca3b  ca4b  b2a  b3a  b4a  ca2a  ca3a  ca4a  b2E  b3E  b4E 

This  is  the  end  of  the  user  input  file  ml407.uif.  Further  examples  may  be 
found  in  Reference  [7] 


3  PIC3D  on  the  Workstation 

To  run  the  simulation  program  PIC3D  issue  the  command  pic3d  with  the  name 
of  the  data  file  as  an  argument.  For  example, 


%  pic3d  inl407.dat 


The  selection  of  output  files  and  their  contents  depends  on  the  specification 
given  in  the  .uif  file.  In  this  instance,  the  following  files  were  generated: 


ml407 . out 
ml407 . grd 
ml407.01 .gst 
ml407-02 .tsd 
ml407-03 .tsd 
ml407-04.grl 
ml407-05 .tsd 


text  output  (mainly  diagnostic)  data 
grid  file  containing  labelhng  graphics  page 
grid  file  containing  scatter  plots 
time  series  data  input  for  MPICTIM 
time  series  data  input  for  MPICTIM 
grid  file  containing  line  plots 
time  series  data  input  for  MPICTIM 
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4  PIC3D  on  the  SP2 

The  main  differences  between  running  PIC3D  on  a  workstation  and  in  multipro¬ 
cessor  runs  on  the  SP2  are  that  the  input  file  has  a  specific  name  input.dat, 
and  that  the  execution  of  PIC3D  leads  to  separate  output  files  for  each  processor. 
The  procedure  for  executing  PIC3D  on  the  SP2  is 

1.  import  the  .dat  file  to  the  file  input.dat, 

2.  set  up  a  PVM3  parallel  machine  on  the  SP2, 

3.  ensure  left-over  files  from  earlier  runs  have  been  deleted, 

4.  execute  pic3d 

5.  merge  output  .tsd  files, 

6.  export  output  files  back  to  the  workstation  for  postprocessing. 

These  steps  are  described  in  detail  in  the  “readme”  file 
“hockney/MAUI-3DPIC/README-MIMD 

on  the  MHPCC  SP2  computer.  A  copy  of  this  “readme”  file  is  attached  as 
Annex  G  of  the  Report  [4] 

5  Using  MPICTIM 

The  version  of  MPICTIM  (see  Technical  Note  [8])  provided  with  the  current  re¬ 
lease  of  3DPIC  differs  from  the  earlier  version  provided  in  that  it  allows  gratic¬ 
ules  on  graphs,  logarithmic  scales,  and  point-and-click  extraction  of  data  {x,y) 
values  from  plots. 

The  procedure  for  getting  displays  is  the  same  as  for  earlier  version.  The 
basic  steps  are  to 

1.  execute  mpictim  to  start  the  Motif  dialogue  panel  and  graph  panel, 

2.  open  the  data  file, 

3.  select  the  plot  type, 

4.  set  plot  ranges, 

5.  plot  curves  and/or  axes, 

6.  save  and/or  erase  plot. 

For  further  details,  refer  to  Technical  Note  [8]. 
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6  Using  DISPAN 

The  dispersion  analysis  postprocessor  DISPAN  allows  the  interactive  interroga¬ 
tion  of  the  .lin  datasets  generated  by  3DPIC.  The  mathematical  description 
of  the  analysis  performed  by  DISPAN  is  described  in  Reference  [6]. 

The  example  given  below  illustrates  how  DISPAN  may  be  used  to  extract 
frequency  and  wavenumber  information  from  datasets  generated  by  PIC3D. 
DISPAN  takes  the  user  through  the  following  sequence  of  operations; 

1.  file  selection 

2.  domain  selection 

3.  field  selection 

4.  spatial  fourier  transformation 

5.  selection  of  time  interval 

6.  time  FFT 

7.  structure  factor  display 

8.  frequency  spectra  at  given  k 

9.  further  selection  options 

These  are  illustrated  below  using  extracts  from  an  interactive  session  using 
DISPAN  on  a  dataset  pcllr02-02.1in  generated  by  PIC3D  from  a  ‘cold-test’ 
run  (ie  a  purely  electromagnetic  calculation)  on  a  five  turn  section  of  an  helix 
TWT.  It  is  assumed  that  the  data  files  vltubel.dat  and  pcllr02-02 .lin  are 
in  the  present  working  directory  (pwd)  and  that  dispan  is  on  the  search  path. 
The  session  begins  by  typing: 

dispan  vltubel.dat 


The  data  file  vltubel.dat  contains  the  following 
tubel.odp 

'NLRES  L  False  for  NEWRUN.True  for  RESET'  / 
tubel 

DISPANX  Version  1.00.00  January  1995 
Dispersion  analysis  tool 

For  use  on  .lin  output  files  from  PIC3D  Version  2.10.00 
Defaults  below  are  for  Tube  1  data 
$NEWRUN 

FRQMIN=  0.0, 

FRQMAX=  35.0, 

BETAMN=-4488 
BETAMX=  4488 
N0CHTD=25 , 
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MCHMIN=10, 

MCHMAX=25 

$END 

The  first  line  gives  the  name  of  the  output  log  file  for  the  session;  in  this  ca^e 
it  is  tubel.odp.  The  second  and  third  line  are  respectively  the  restart  flag 
(dummy  in  this  instance)  and  reference  label  tubel.  The  next  four  lines  are 
labelling  for  the  output  log. 

The  main  purpose  of  the  data  file  vltubel .  dat  is  to  provide  default  ranges 
for  wavenumbers  and  frequencies  on  the  plots.  Choosing  appropriate  values 
avoids  a  lot  of  resetting  of  plot  ranges  during  the  interactive  session.  In  this 
cases,  we  are  looking  at  data  from  a  helix  with  pitch  p  =  1.410“^m,  and  are 
interested  in  data  in  the  first  zone,  so  a  wavenumber  range  [—K,K],  where 
K  =  27rfp  =  4488m~^  and  frequency  range  [0,  35  GHz]  make  reasonable  choices. 
Wavelength  ranges  are  set  by  parameters  BETAMN,  BETAMX,  and  frequency  ranges 
are  set  by  FRQMIN  and  FRQMAX.  The  number  of  uniformly  spaced  contour  heights 
spanning  the  data  is  given  by  NOCHTD,  and  the  range  of  contour  heights  plotted 
is  given  by  [MCNMIN,MCHMAX]. 

File  Selection.  The  first  interaction  is  the  selection  of  the  .  lin  dataset.  The 
program  prompts  for  the  root  of  the  file  name,  and  asks  for  verification  if  it 
finds  it  in  the  pwd. 

Please  enter  the  root  of  the  .lin  file  name 
pcllr02-02 

Open  data  file?  (y|n)[y]  name:  pcllr02-02.1in 

y 

Grid  file  called  pcllr02.gdp 

If  the  data  is  not  found,  the  program  will  repeat  prompting  until  a  valid  file 
is  found.  If  it  is  found,  then  DISPAN  announces  the  name  it  generates  for  the 
output  grid  file  (pcllr02.gdp  in  this  example). 

Domain  Selection.  A  dataset  may  contain  line  data  along  domains,  then  a 
menu  is  displayed  and  the  users  selects  the  one  he  wants.  In  this  example,  only 
one  domain  is  present  and  so  it  is  automatically  selected. 

DOMAIN  SEARCH  COMPLETED 

Domain  Selection  Menu 

only  one  option  available 
Option  selected:  cixial  distcince  (m) 

Field  Selection.  Each  domain  may  contain  data  for  several  fields.  For  exam¬ 
ple,  a  domain  could  be  a  line  along  the  axis  of  the  device  on  which  components 
of  magnetic  and  electric  field  are  accumulated.  Here,  only  one  field,  the  scalar 
helix  to  body  tube  voltage  is  stored,  so  again  the  selection  defaults  to  the  only 
one  available. 
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Field  Selection  Menu 
only  one  option  available 

Option  selected:  radial  voltage  (helix  -  body  tube) 
Now  loading  dataset  -  please  wait 
radial  voltage  (helix  -  body  tube) 


Spatial  Fourier  Transform.  Once  the  data  is  read  in,  it  is  spatially  fourier 
transformed.  The  temporal  transform  is  designed  for  use  with  a  small  number 
of  periods  of  the  fundamental  signal  (~  1-2),  and  this  necessitates  the  choice 
of  interval  for  the  transform  to  fit  the  period  of  the  data  to  get  good  spectral 
results.  The  alternative,  using  a  cosine  bell  and  zero  padding  to  eliminate  spec¬ 
tral  aliasing  was  not  adopted  as  it  requires  a  much  larger  number  of  periods  of 
oscillation  in  the  data  to  get  good  spectral  resolution.  The  method  for  select¬ 
ing  the  data  time  interval  implemented  is  a  visual  point-and-click  approach,  as 
described  below. 

Selection  of  Time  Interval.  The  following  extract  plots  the  time  evolution 
of  the  k  =  1  harmonic  (ie  wavenumber  (3  =  2'KklL  =  211  jL,  where  L  is  the 
domain  arc  length  =  bp  for  this  example).  Where  yes/no  answers  show  default 
option  (eg  Cn]),  then  simply  pressing  return  takes  that  default. 

Select  wavenumber  index  from  range  0  to  60 

1 

Set  plot  ranges?  [n] 

Time  Filtering?  [n] 

Extract  Data  Values?  [n] 
n 

The  result  of  this  step  is  the  plot  shown  in  figure  2.  When  viewed  on  the 
screen,  the  red  curve  corresponds  to  the  cosine  harmonic,  and  the  blue  to  the 
sine  harmonic. 

It  is  obvious  from  figure  2  that  the  low  frequency  component  is  difficult  to 
identify,  so  we  replot  the  data,  this  time  passing  it  through  a  low  pass  filter  to 
make  the  lower  frequency /longer  wavelength  component  easier  to  identify.  The 
following  extract  from  the  terminal  session  shows  the  user  select  a  plot  range 
for  amplitude  from  -30  to  30,  and  a  width  11  for  the  time  filter.  The  resulting 
plot  is  shown  in  figure  3. 

Replot  same  data?  [n] 

y 

Set  plot  ranges?  [n] 

y 

Harmonic  amp  min  =  -268.24  max  =  285.23  OK  y/n? 

n 


16 


J  W  Eastwood,  W  Arter,  N  J  Brealey 


Harmonic  Amplitudes  1  aea  T.chB<.i«f7 

Cnlhatn 

DISPAK  Vertten  1.00.00  iiftairr  1»S5 


Figure  2:  The  time  evolution  of  the  k  =  1,P  =  877.6m  ^  harmonic 


Input  new  min  and  max 
-30  30 

Harmonic  amp  min  =  -30.000  max  =  30.000  OK  y/n? 

y 

Time  range  min  =  0.40045E-02  max  =  0.99712  OK  y/n? 

Time  Filtering?  [n] 

y 

Input  filter  length  (1-100) 

11 


The  fundamental  mode  in  figure  3  is  now  easy  to  identify.  Answering  y 
to  the  Extract  Date  Values  query  activates  the  cursor,  and  pressing  the  left 
mouse  button  at  the  cursor  position  (marked  by  crosses  on  figure  3)  prints 
values  in  the  dialogue  screen.  In  this  example,  six  oscillation  periods  have  been 
chosen.  Pressing  the  right  mouse  button  terminates  the  cursor  interaction,  and 
answering  no  to  the  next  two  prompts  brings  up  the  selection  of  the  time  range. 
Here  the  values  obtained  at  the  cursor  have  been  used  and  confirmed. 

Extract  Data  Values?  [n] 

y 

Mouse  left  button  for  data,  right  for  done 
Time  =  0.10841  Amplitude  =  0.16281E-01 

Time  =  0.95892  Amplitude  =  -0.10914 

Replot  same  data?  [n] 


Another  Harmonic?  [n] 
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Harmonic  Amplitudes  1  *ea  Technoi«*7 
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Figure  3:  A  time  filtered  plot  for  the  time  evolution  of  the  £  =  1  harmonic 

TIME  RANGE  min  =  0.40045E-02  max  =  0.99712  OK  y/n? 

n 

Input  new  min  and  max 
.1084  .9589 

TIME  RANGE  min  =  0 . 10840  max  =  0 . 95890  OK  y/n? 

y 

Time  FFT.  Once  the  time  range  is  entered,  DISPAN  performs  a  periodic 
spline  interpolation  to  a  set  of  2^  uniformly  spaced  points  spanning  the  time 
range,  and  then  performs  a  periodic  fourier  transform. 

Structure  Factor  Display.  The  following  dialogue  shows  the  bidirectional 
spectrum  selected.  The  range  of  values  of  S  are  spanned  uniformly  by  the 
NOCHTD  contours  (=  25  in  this  case)  and  the  user  is  prompted  for  the  subset  of 
these  heights  to  be  plotted. 

bidirectional  spectrum?  [y] 

y 

Reset  plot  ramge?  [n] 

Amplitude  normalised  plot?  [n] 


min  cont  =  10 

/ 


max  cont 


/ 


25 
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Structure  factor  itchnouiy 
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Figure  4:  The  structure  factor  plot.  The  peaks  in  the  contour  plots  show 
locations  of  points  on  the  ujj P  diagram 

Extract  Data  Values?  [n] 
n 

The  result  is  plotted  in  figure  4.  It  is  usually  easier  to  pick  out  the  spectral 
peaks  in  structure  factor  plots  where  the  maximum  at  each  wavenumber  is 
normalised  to  unity.  The  next  dialogue  extract  and  figure  5  shows  this: 

Replot  same  data?  [n] 

y 

Reset  plot  range?  [n] 

Amplitude  normalised  plot?  [n] 

y 

min  cont  =  10 

20 

min  cont  CHANGED  TO  20 

max  cont  =  25 

/ 

Extract  Data  Values?  [n] 

y 

Mouse  left  button  for  data,  right  for  done 
beta  =  885.43  rad/m,  f  =  7.0337  GHz 

beta  =  1835.1  rad/m,  f  =  14.082  GHz 

Replot  same  data?  [n] 
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Figure  5:  The  normalised  structure  factor  plot 

The  /  and  /?  values  can  be  extracted  directly  from  the  structure  factor  plots  by 
clicking  the  mouse  at  the  cursor  position,  as  illustrated  in  the  above  extract,  or 
can  be  measured  more  precisely  from  the  spectral  plots  at  given  wavenumbers. 

Frequency  Spectra.  After  completing  the  structure  factor  dialogue,  DISPAN 
moves  on  to  plotting  frequency  spectra.  The  following  dialogue  overplots  the 
spectrum  for  left  going  waves  (A:  =  1,  solid  curve)  and  right  going  waves  {k  — 
— 1,  broken  curve)  at  wavenumbers  (3  =  897.6m~^,  and  then  extracts  /?  and  / 
using  the  cursor. 

Select  wavenumber  index  from  range  -60  to  60 

1 

Set  plot  ranges?  [n] 

Extract  Values?  [n] 
n 

Replot  same  data?  [n] 
n 

Another  Harmonic?  [n] 

y 

Select  wavenumber  index  from  range  -60  to  60 
-1 

Set  plot  ranges?  [n] 

Overplot?  [n] 

y 

Extract  Values?  [n] 


y 
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Figure  6:  Frequency  spectrum  plots  at  /3  =  897.6m“^.  The  solid  and  dashed 
curves  and  respectively  for  the  left  and  right  going  waves.  The  peaks  correspond 
to  the  fundamental  and  /?  =  3590. space  harmonic. 


Click  cursor  to  right  of  plot  to  end 
Mouse  left  button  for  data,  right  for  done 
Frequency  =  7.0007  Amplitude  =  1.0012 

Frequency  =  22.320  Amplitude  =  0.85528 

Replot  same  data?  [n] 
n 

The  plot  corresponding  to  this  dialogue  is  shown  in  figure  6.  Further  wavenum¬ 
ber  spectra  can  be  similarly  be  generated  and  plotted  on  the  same  or  separate 
graphs. 


Further  Selections.  Answering  no  to  the  Another  harmonic  query  com¬ 
pletes  the  interrogation  of  the  current  domain/field  selection.  For  domains 
with  multiple  fields  and  datasets  with  multiple  domains,  the  dialogue  returns 
to  the  appropriate  menu.  In  this  example,  the  only  further  option  is  another 
data  file.  Selecting  no  completes  the  session. 


Another  Harmonic?  [n] 

Another  data  file?  [y] 
n 

A  further  example  of  the  use  of  the  dispersion  analysis  postprocessor  is  given 
in  Section  5.5  of  Reference  [6]. 
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7  Viewing  Graphics  using  GHOST 

The  simplest  GHOST  tool  for  viewing  the  grid  graphical  files  is  the  command 
line  interface  tool  xghost.  For  example,  to  view  the  frames  in  file  ml407-01 .  gst 
sequentially,  simply  type 

xghost  ml407-01.gst 

and  click  on  the  graphics  window  to  advance  the  frame. 

To  select  specific  frames,  and  to  overplot,  scale,  rotate,  etc,  xghost  must  be 
used  in  interactive  mode: 

xghost  -int 

CMND?  pict  ml407-01.gst  3  5 


CMND?  gr 

The  GHOST  graphical  (grid)  files  can  be  converted  into  HPGL2,  CGM  or 
POSTSCRIPT  (and  many  other  formats)  for  further  processing  or  printing. 
For  example,  to  generate  monochrome  Postscript  output  in  portrait  orientation 
in  output  file  sample.ps  from  the  input  file  sample. grd,  type 

postsc  -gpor  -gfil  sample.ps  sample. grd 

For  colour  Postscript  or  HPGL2  output,  substitute  respectively  post  cl  or 
hpgl2  for  postsc  in  the  above  example. 

For  more  details  on  xghost,  postsc,  etc  see  the  GHOST  User  Manual  [3] 
section  entitled  “GHOST  for  UNIX”. 
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Abstract 

The  concepts  underlying  our  approach  to  mesh  generation  are  outlined, 
and  then  a  detailed  specification  is  presented.  This  document  extends 
earlier  notes  issued  on  this  topic  [3,  4,  5]  to  include  the  interface  to  produce 
parameters  that  cause  PiC3D  to  perform  simulations  of  particle  beams  and 
secondary  emission. 


1  INTRODUCTION 

This  Note  describes  the  inputs  acceptable  to  the  PEGGIE  (Particle  Electro¬ 
magnetic  General  Geometry  InterfacE)  preprocessor  for  the  3D  PIC  simulation 
package.  The  basic  mesh  generation  function  of  PEGGIE  has  now  been  enhanced 
and  extended  to  the  point  where  it  can  produce  datasets  that  will  cause  PIC3D 
to  simulate  complex  physics,  such  as  electric  fields  on  resistive  walls  and  particle 
emission.  However  only  discussion  of  the  grid  generation  issues  appears  here. 
For  the  additional  topics,  this  publication  serves  as  only  a  user  manual. 

The  complexity  of  interfaces  to  existing  mesh  generation  software  has  led 
us  to  produce  a  simpler  interface  tailored  to  the  needs  of  the  program  under 
development.  The  concept  underlying  our  approach  is  that  the  device  to  be 
modelled  is  made  by  joining  together  ‘parts’,  with  appropriate  coding  to  gener¬ 
ate  the  mesh  and  associated  information  for  each  class  of  part.  One  important 
advantage  is  that  ‘appropriate’  coding  could  just  be  that  which  converts  the 
output  of  another  mesh  package  into  a  format  consistent  with  our  new  soft¬ 
ware. 

For  reasons  of  simplicity  and  portability,  input  to  PEGGIE  takes  the  form 
of  a  file  containing  text.  Previous  experience  highlights  the  value  of  using  tags 
within  the  file,  that  take  the  form  of  two  character  words  appearing  at  the  start 
of  an  item.  The  tags  serve  to  tell  the  processor  the  format  of  subsequent  data. 
The  following  discussion  indicates  the  formats  that  may  be  required. 

First,  let  us  observe  that  each  class  of  part  must  have  a  unique  name  since 
it  may  correspond  to  a  piece  of  code,  which  good  programming  practice  implies 
must  be  a  subroutine.  In  this  case  there  will  be  parameters  to  be  set,  e.g. 
the  dimensions  of  a  block  and  the  number  of  mesh-points.  Several  different 
parts  may  belong  to  the  same  class,  hence  some  sort  of  marking  is  needed  to 
distinguish  one  part  from  another.  Moreover,  there  are  effectively  two  sorts  of 
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class,  not  only  classes  which  correspond  to  subprograms  of  the  mesh  software, 
but  also  those  that  are  determined  dynamically  e.g.  by  reading  the  output  of 
another  mesh  generator.  Such  output  may  be  extremely  verbose,  hence  it  is 
desirable  to  be  able  to  tell  the  code  to  look  in  a  disk  file  for  the  data. 

The  characteristics  required  of  the  most  general  tagged  record  are  thus 
that  it  should  contain  a  label,  a  class  name,  some  way  of  specifying  whether 
a  disk  file  is  to  be  searched,  and  then  some  parameters.  Experience  again 
suggests  that  all  but  very  small  sets  of  parameters  are  best  specified  using  the 
NAMELIST  format  of  input.  NAMELIST  formats  are  defined  as  part  of  the 
FORTRAN  90  standard[2],  and  most  FORTRAN  77  compilers  of  which  we  are 
aware  accept  NAMELIST,  with  slight  variations  that  can  be  easily  corrected  for. 
NAMELIST  is  not  so  useful  if  all  parameters  in  a  set  have  to  be  specified,  hence 
both  NAMELIST  and  ordinary  lists  will  be  permitted.  The  resulting  record 
structure  enables  compactness  of  representation  coupled  with  great  flexibility 
and  readability. 

The  key  concepts  underlying  our  approach  to  mesh  generation  are  further 
expounded  in  Section  2.  Section  3  describes  the  three  basic  input  formats 
required,  of  which  the  first  or  standard  is  by  far  the  most  important.  Section 

4  contains  a  detailed  specification  for  input  to  PEGGIE,  and  the  final  section 

5  describes  the  classes  available.  Section  3  also  includes  a  description  of  the 
interface  to  the  production  of  parameters  that  control  output  from  a  P1C3D 
run. 


2  KEY  CONCEPTS 

As  already  mentioned,  a  device  is  conceived  of  being  made  up  of  parts.  Let  us 
think  of  a  part  as  relating  to  a  single  multiblock  or  uniblock.  For  book-keeping 
purposes  it  is  necessary  for  every  uniblock  to  have  a  unique  label.  However, 
just  as  say  many  cars  have  four  identical  wheels,  so  several  uniblocks  may 
correspond  to  the  same  part.  Defining  a  uniblock  thus  involves  (i)  giving  the 
uniblock  a  label  and  (ii)  specifying  the  part  to  which  it  relates,  by  giving  the 
part  description. 

The  part  description  is  composed  of  two  labels,  since  it  is  useful  to  be  able 
to  separate  the  part  geometry  from  the  part  physics,  i.e.  to  be  able  to  specify 
permittivities  and  conductivities  etc.  independently  of  the  shape  of  the  part. 
The  part  physics  is  specified  either  by  giving  a  class  and  associated  parameters, 
or  as  an  entity  on  disk.  The  part  geometry  may  be  specified  similarly. 

Having  specified  the  parts,  or  more  precisely  the  multiblocks,  it  is  necessary 
to  describe  how  they  are  put  together.  If  two  blocks  A  and  B  have  a  common 
face,  then  it  is  only  necessary  to  give  the  information  that  a  particular  face  of 
block  A  maps  to  a  face  in  block  B.  The  convention  for  labelling  faces  is  indicated 
in  Fig  1  which  shows  how  the  labels  N,  S,  E,  W,  U  and  D  (respectively  North, 
South,  East,  West,  Up  and  Down)  relate  to  the  coordinates  used  within  a  block 

If  blocks  only  meet  over  part  of  a  face  then  it  is  necessary  to  specify  which 
fraction  of  each  block’s  face  is  involved.  Incorporating  ideas  from  Ref  [2],  con¬ 
nectivity  is  specified  via  surface  patches.  Further,  it  is  helpful  to  consider  the 
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Figure  1:  Illustrates  the  labelling  of  faces  of  a  multiblock.  Position  xq  is  the 
origin  of  the  block  co-ordinates  and  vectors  xi,  X2  and  X3  indicate  the  1,  2  and 
3  co-ordinate  directions  respectively. 


co-ordinate  system  in  which  the  block  maps  to  the  unit  cube.  (Such  a  system  is 
obtained  by  a  straightforward  contraction  from  the  system  proposed  in  Ref  [2]). 
The  unit  cube  system  is  then  used  to  measure  the  extent  of  the  surface  patch. 
The  surface  patch  is  specified  by  giving  the  face  label  and  co-ordinates  of  a 
pair  of  opposite  corners  (the  ‘O’  and  ‘X’  points).  Note  the  implication  that  the 
patch  shape  in  physical  space  is  effectively  determined  by  the  surface  in  which 
it  lies. 

To  specify  connectivity  in  the  input  file,  a  list  rather  than  a  NAMELIST 
is  appropriate,  since  we  adopt  the  convention  that  for  each  block,  connectivity 
information  for  every  face  must  appear.  Although  this  implies  a  factor  of  two 
redundancy  in  the  input,  since  clearly  the  same  patch  information  appears  in 
the  two  entries  for  the  adjacent  blocks,  validation  of  the  input  is  considerably 
simplified. 

It  is  desirable  also  to  specify  the  order  in  which  the  parts  are  assembled. 
The  main  reason  is  the  need  to  determine  how  many  elements  are  used  in  each 
surface  patch.  It  is  easy  to  see  that  conflicts  can  occur.  For  example  in  Block 
A  there  may  have  to  be  say  12  x  12  element  faces  in  the  face  common  to  Block 
B,  because  Block  A  has  been  gridded  using  a  foreign  mesh  generator,  but  the 
chosen  gridding  for  Block  B  might  be  16  xl6  x  16  elements.  The  order  of 
assembly  can  be  used  to  resolve  conflicts  by  preferring  data  associated  with  the 
block  that  appears  earlier  in  the  ranking.  In  the  example  shown,  if  A  appears 
before  B,  then  the  code  checks  to  see  whether  a  12  x  12  x  12  gridding  is  possible 
for  Block  B,  which  will  usually  be  the  case  if  B  is  specified  as  a  subprogram 
class.  If  B  appears  before  A  then  the  conflict  cannot  be  resolved  and  an  error 
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message  should  result. 

The  above  example  serves  to  introduce  the  concept  of  elastic  grid  sizes 
ni  X  n2  X  ns  where  Uj  is  the  number  of  elements  in  a  coordinate  direction  i. 
For  most  blocks,  the  associated  are  parameters  that  can  be  changed  to  try 
to  resolve  conflicts.  The  convention  adopted  is  that  instead  of  using  positive 
integers  for  the  rii  in  these  blocks,  a  negative  integer  is  used  for  each  co-ordinate 
direction  in  which  the  grid  size  is  effectively  arbitrary.  It  is  worth  noting  that 
even  when  all  blocks  have  been  collected  some  rii  may  remain  elastic,  and 
means  are  provided  for  defining  these.  The  default  is  to  replace  negative  rij  by 
their  absolute  values,  so  that  in  effect  negative  nj  suggest  grid  sizes  but  do  not 
prescribe  them. 

3  BASIC  INPUT  FORMATS 

The  Section  begins  by  describing  the  standard  tagged  input  format  then  goes 
on  to  outline  other,  simpler  tagged  formats.  Detailed  specifications  for  each  tag 
are  given  in  Section  4. 

3.1  Standard  Format 

The  standard  format  of  a  tagged  record  as  used  in  Section  4  is  written: 

QQulabelu  Chowstored=] classu [parlupar2 . . . ]  . 

Labels  are  case  sensitive  and  should  not  contain  spaces,  commas  or  colons. 
If  the  classname  contains  spaces,  these  may  be  replaced  by  underscore  or  the 
entire  name  should  be  enclosed  in  quotes. 

To  explain  the  above,  note  first  the  convention  that  items  appearing  within 
square  brackets  are  optional.  The  three  compulsory  items,  each  separated  by 
at  least  one  space  “□  ”,  are  QQ  which  is  the  generic  form  for  a  tag,  a  label 
which  does  not  occur  in  any  other  record  with  the  same  tag,  and  a  classname. 
An  example  of  such  a  simple  record  is: 

BGupartlguregular_cubic_lattice 

This  creates  a  block  geometry  entry  that  is  referenced  by  the  label  partlg  and 
described  as  belonging  to  the  class  regular_cubic_lattice  so  that  e.g.  there 
exists  a  subroutine  CUBREG  to  generate  the  particular  geometry. 

Parameters  that  determine  partlg  within  the  class  regular_cubic_lattice 
are  specified  in  subsequent  lines  of  input,  using  NAMELIST  as  defined  in  the 
standard[l].  Thus  the  complete  entry  for  partlg  also  includes,  e.g. 

&CUBREG 

RXMAX  =0.1,  RYMAX  =2.0,  RZMAX  =3.0/ 

that  specifies  the  dimensions  of  the  cuboid  region  (in  metres) . 

The  optional  entries  pari,  par2,  ...  are  special  parameters  that  can  be  set 
without  recourse  to  the  verbosity  of  NAMELIST.  Their  meaning  depends  on 
the  tag.  For  BG  they  are  the  nj,  in  order  f  =  1, 2, 3,  defining  the  grid  size.  Thus 
a  more  complicated  entry  might  be 

BGupartlLjregular_cubic_latticeu-20u“60u“40 
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The  use  of  minus  denotes  the  size  is  elastic  (see  Section  2),  but  that  a  suggested 
mesh-size  for  the  part  is  20  x  60  x  40  elements. 

As  its  name  suggests  the  optional  howstored  entry  indicates  how  the  infor¬ 
mation  is  stored.  If  it  is  omitted  a  namelist  normally  follows  as  in  the  example. 
However  other  values  are  possible,  depending  on  the  class  and  tag.  All  tags 
allow  howstored  to  take  the  value  file,  indicating  the  file  with  the  name  class 
is  to  be  searched  for  the  remaining  information  associated  with  the  entry.  The 
file  class  will  itself  contain  an  entry  of  the  above  form. 

Values  for  howstored  that  may  be  allowed  depending  on  the  tag  are  number 
and  metric.  Number  implies  that  the  block  geometry  is  specified  as  a  set  of 
numbers  representing  e.g.  the  co-ordinates  of  mesh-points.  Metric  indicates 
that  the  metric  information  needed  for  the  timestepping  calculation  is  already 
available.  The  above  serve  directly  or  indirectly  to  define  new  classes  of  geom¬ 
etry. 


3.2  List  of  Labels 

There  are  basically  two  simpler  formats  for  input.  One  consists  of  a  tag  followed 
by  a  list  of  labels,  constituting  a  complete  entry.  This  is  used  e.g.  to  define 
blocks  in  terms  of  predefined  block  geometries  and  physics  thus 
BLubl lupart Igupart Ip 

and 

BLubl2usameasubll . 

Note  that  sameas  has  a  special  meaning  and  should  not  therefore  be  used  as 
a  block  label.  Any  blocks  defined  without  use  of  a  sameas  can  also  be  thought 
of  as  “block  types” .  The  same  format  is  used  to  specify  the  order  of  assembly 
of  blocks. 


3.3  Connectivity  Format 

The  second  simpler  input  format  is  used  to  define  connectivities.  The  tagged 
line  contains  only  a  label  (in  addition  to  the  tag).  Subsequent  lines  contain  the 
block  connectivity  information  as  series  of  entries  (one  to  a  line)  in  the  format: 


facel[{xi^,X2^){x\^,xl^)]  =  block  :  face2[{xf^,X2^){xi^,X2^)] 

terminated  by  a  line  containing  only  the  tag  EN.  Omitting  the  quantities  in 
square  brackets  is  equivalent  to  taking 

(xf  )  =  (0,0)(l,l),i  =  1.2. 

The  X*®  and  are  the  co-ordinates  of  the  ‘O’  and  the  ‘X’  points  defining 
a  patch  on  the  face  facel  where  i  =  1  refers  to  the  block  of  which  the  label 
appears  on  the  tagged  line  and  i  =  2  refers  to  other  blocks,  the  labels  of  which 
appear  immediately  to  the  right  of  the  equals  sign. 
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4  DETAILED  INPUT  SPECIFICATION 

The  allowed  tags  are  assigned  one  of  three  input  formats.  The  tags  BG,  BP 
and  PP  are  used  to  define  respectively  the  geometry  of  a  block,  the  physics  of 
a  block,  and  physical  data  associated  with  a  surface  patch  (including  boundary 
conditions).  The  tag  CU  is  used  to  define  a  curve  within  the  global  geometry. 
The  tag  DE  is  used  to  set  default  values  for  parameters,  and  SF  is  used  to  set 
parameters  that  otherwise  have  no  associated  tag.  The  input  associated  with  all 
these  tags  (and  FL,  PO  and  SC  to  be  defined  below)  has  the  standard  format. 

The  tags  BL  and  PA  are  used  to  define  respectively  a  block  in  terms  of 
its  geometrical  and  physical  attributes  and  a  patch  in  terms  of  its  physical 
attributes.  Tag  OR  is  used  to  specify  the  order  in  which  blocks  should  be 
connected.  The  input  associated  with  these  tags  has  the  list  of  labels  format. 

The  tag  BC  is  used  to  define  block  connectivity  information.  The  informa¬ 
tion  in  Section  3.3  constitutes  a  detailed  input  specification  for  this  tag,  thus 
no  further  description  appears  here. 

The  tag  PS  is  used  to  define  a  plot  set,  i.e.  data  that  is  to  be  output  by  the 
PIC3D  for  subsequent  analysis  and  plotting.  A  plot  set  is  defined  in  terms  of 
labels  that  identify  the  fields  to  be  output,  the  locations  at  which  their  value 
is  to  be  output,  and  the  corresponding  times.  The  first  two  are  defined  by  list 
of  labels  tagged  with  FS  for  fields  and  DS  for  the  set  of  domains,  and  the  last 
by  tag  TS  which  has  the  standard  format.  The  plot  output  of  fields  may  be 
controlled  more  tightly  by  using  the  FL  tag  and  its  associated  namelist.  The 
domain  set  is  a  list,  either  of  domains  specified  by  the  list  of  labels  tagged  by 
DO,  or  of  subdomains.  Each  domain  is  made  from  subdomains  that  are  defined 
by  tag  SU  that  also  has  the  standard  format.  The  importance  of  the  domain 
concept  is  that  fields  are  summed  over  its  constituents. 

The  Table  provides  a  summary  of  formats  for  the  different  tags. 

4.1  Tags  in  Standard  Format 

4.1.1  BG,  BP,  CU  and  PP 

Input  associated  with  the  tag  BG  has  been  specified  in  Section  3.1.  Much 
the  same  detailed  specification  applies  to  tag  BP.  The  only  major  differences 
is  that  the  metric  information  now  relates  to  permittivity,  permeability  and 
conductivity,  rather  than  to  position.  As  might  be  expected,  the  associated 
namelist  names  also  differ.  Tag  PP  input  is  very  similar,  with  three  exceptions, 
namely  (i)  that  the  special  parameters  are  now  only  ni  and  n2  since  surfaces 
are  two-dimensional,  (ii)  that  it  is  inappropriate  to  have  direct  input  of  metric 
terms  as  a  howstored  option,  and  (iii)  the  namelist  names  are  different.  Tag  CU 
enables  a  curve,  possibly  closed,  to  be  defined  by  inheritance  from  a  preexisting 
geometry  and  usually  by  extension  of  a  line  specified  in  one  block. 

4.1.2  DE,  FL,  PO,  SC  and  SF 

All  these  tags  have  no  special  parameters  and  allow  howstored  only  to  take  the 
value  file.  If  howstored  is  not  set,  then  default  values  are  set  for  variables  in  the 
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Table  1:  Tag  Table 


Tag 

Brief  Description 

Format + 

Special  Parameter 

Notes* 

BC 

Block  Connectivity 

C 

- 

- 

BG 

Block  Geometry 

Ni,N2,N3 

f  n  m 

BL 

BLock  definition 

- 

3  labels 

BP 

Block  Physics 

NuN2,N, 

f  n  m 

CU 

CUrve  definition 

- 

- 

DE 

DEfaults  (namelists) 

s 

- 

f 

DO 

DOmain 

L 

- 

any  number  of  labels 

DS 

Domain  Set 

L 

- 

any  number  of  labels 

FL 

FieLd  data 

S 

- 

f 

FS 

Field  Set 

- 

any  number  of  labels 

OR 

ORdering  of  blocks 

- 

any  number  of  labels 

PA 

PAtch  definition 

- 

3  labels 

PO 

Parameter  StOrage 

s 

- 

f  n 

PP 

Patch  Physics 

s 

Ni,N2,Ni 

f  n 

PS 

Plot  Set 

L 

- 

4  labels 

SC 

Signal  Component 

s 

- 

f 

SD 

Secondary  Domains 

L 

- 

any  number  of  labels 

SF 

Set  Free  parameters 

S 

- 

f 

SG 

SiGnal 

L 

- 

- 

SP 

SPecies  definition 

S 

Ni  -  N5/R1  -  Rs 

f  n 

SU 

subdomain 

S 

Ni  —  Ns,Ri  —  Re 

f 

TS 

Time  Set 

s 

Ni  -  Ne/Ri  -  Re 

f 

*f,  n  and  m  axe  abbreviations  for  the  allowable  howstored  options  file,  numbers 
and  metric. 

+S,  L  and  C  denote  the  standard,  the  list  of  labels  and  the  connectivity  formats 
respectively 
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namelist  that  follows  in  the  case  of  DE. 

For  PO,  parameters  are  set  defining  the  initial  magnetic  field,  initial  current, 
source  current,  particle  beam,  initial  electric  field,  external  magnetic  field  or 
external  electric  field,  depending  whether  the  label  is  set  to  bf  ield,  current, 
j source,  pbeam,  efield,  bext  or  eext  respectively.  The  parameters  defined 
are  a  subset  of  NAMELIST  SF,  namely 


AMODE(3) 

BUNI(3) 

CDENO 

CMODE(2) 

CPO 

FMODE 

NINIT 

NMODE(3) 

RPHSHF(3) 

ZMODE 


box  size  for  modes 
uniform  initial  B 
beam  number  density 
mode  amplitudes 
beam  momentum  in  z 
frequency  of  mode 
initial  field  type 
mode  numbers 
phase  shifts  (in  radians) 
impedance  of  mode 


Negative  NINIT  set  an  initial  current  that  lasts  for  the  first  timestep  only  (a 
“ping”),  the  spatial  dependence  being  determined  by  the  absolute  value  as 
follows: 

1.  an  initial  uniform  magnetic  field 

2.  an  initial  cylindrical  TE  mode 

3.  a  cylindrical  TM  mode 

4.  a  rectangular  TE  mode 

5.  a  rectangular  TM  mode 

6.  a  magnetostatic  (PPM)  mode. 

7.  a  spiky  mode,  a  sum  over  four  axial  harmonics  of  a  TM  mode 

8.  a  helical  ping  of  z  component 

9.  a  cylindrical  ping  of  z  component 

The  field  components  are  set  by  BUN  I  and  mode  parameters  are  set  by  variables 
ending  in  MODE.  Clearly  only  simple  device  geometries  are  allowed,  and  the 
3-coordinate  must  correspond  to  z. 

For  FL,  NAMELIST  FL  allows  the  setting  of  the  variable  LFSKIP  to  control 
particle  skip  length  for  the  field  given  by  CFLNAM  and  other  parameters  as 
listed 

CFLNAM* 64  diagnostic  field  name 

FMAX  diagnostic  field  maximum 

FMIN  diagnostic  field  minimum 
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FUNIT 

diagnostic  field  unit 

FXMAX 

diagnostic  x-field  maximum 

FXMIN 

diagnostic  x-field  minimum 

FXUNIT 

diagnostic  x-field  unit 

FXZERO 

diagnostic  x-field  zero 

FZERO 

diagnostic  field  zero 

LFCOLO 

diagnostic  field  colour 

LFSKIP 

diagnostic  field  particle  skip 

For  SC  a  component  of  a  signal  is  specified  via  NAMELIST  SIGNAL  which 
contains  the  variables 


FREQUENCY 

DURATION 

SIGNAL_FORM*8 

PHASE 


signal  component  frequency  in  Hertz 
component  duration  in  s 
identifies  form  of  signal  component 
phase  of  signal  component  (degrees) 


The  allowed  SIGNAL.FORMs  are  ‘CONSTANT’,  ‘LINEAR’,  ‘RISE’, ‘PLATEAU’ 
and  ‘S-SHAPE’.  For  SF,  depending  on  class,  parameters  are  set  that  otherwise 
have  no  associated  tags. 

4.1.3  TS  and  SU 

The  time  set  specifies  the  times  at  which  output  is  required; 

TSutimeset-nameuclass  [ystart-timeuend-timeuoutput- 
int ervaluaverage-t imeut ime-zerout ime-unit] . 

The  time -unit  is  the  negative  of  the  base  ten  exponent  of  the  scale  factor 
for  the  times,  i.e.  9  implies  time  measured  in  nanoseconds.  If  the  time-unit 
is  zero  and  all  other  times  are  integral,  then  the  unit  of  time  is  taken  to  be  the 
timestep.  If  no  times  are  set  then  output  will  take  place  at  every  timestep  to  a 
maximum  set  by  NRUN.  The  value  of  class  is  arbitrary. 

The  subdomain  specifies  a  cuboidal  subset  of  a  block 
SUuSubdomain-nameublock-name [uduOux] 

The  block-name  is  the  name  of  a  block,  o  and  x  are  the  positions  of  the  corners 
of  the  subdomain  in  coordinates  on  [0,1]^.  If  o  and  x  are  omitted  the  entire 
block  is  assumed,  d  is  a  triple  of  the  integers  (1,  2,  3)  which  gives  directions  for 
evaluating  components  and  integrals,  d  (1)  gives  the  direction  for  line  integrals, 
ie  the  integral  is  taken  in  the  direction  Sd(i),  and  if  the  subdomain  extends 
normal  to  d(l),  a  set  of  line  integrals  one  for  each  element  in  the  normal  plane 
will  be  computed.  For  surface  integrals,  the  area  element  is  proportional  to 
dfd(i)dxd(2)  and  a  set  may  also  be  computed.  If  d  is  omitted  then  d(i)  =  i,  i  = 
1,  2,  3.  Experts  may  take  the  d(i)  negative  and  set  d(3).  There  are  no  namelists 
corresponding  to  either  TS  or  SU. 
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4.1.4  SP 

Tage  SP  defines  a  species 

SPuspecies-nametjclassuinassuChargeun'uinber- 

particlesuexponentystorageutime-step 

where  the  species  mass  is  in  units  of  the  electron  mass,  charge  is  units  of  the 
absolute  charge  of  an  electron  and  number-particles  is  the  number  of  particles 
on  a  species  superparticle,  the  exponent  of  which  is  given  separately.  The 
variable  storage,  if  not  greater  than  100,  is  the  number  of  particles  per  element, 
and  time-step  is  in  units  of  the  electromagnetic  timestep.  For  example,  the 
electron  species  line  might  read 

SPuelectronuanyulu“lulu9u8u2 

if  there  are  10^  charges  on  a  superparticle,  8  particles  are  expected  per  element 
and  the  subcycling  parameter  NSUBS  =  2. 

Further  particle  data  may  be  specified  using  the  NAMELIST  POLE,  for  the 
classes  below  and  extra,  see  Section  5.3. 

4.2  List  of  Labels  Tags 

4.2.1  BL  and  PA 

Tags  BL  and  PA  define  a  multiblock  and  a  surface  patch  respectively.  The 
format  for  BL  is  specified  in  Section  3.2,  and  that  for  PA  differs  only  in  that 
the  label  following  the  patch  identifier  is  usually  sameas,  thus  an  example  of  a 
record  is: 

PAupalusameasuperfect_conductorl 

where  the  patch  physics  associated  with  label  perfect-conductor  1  has  already 
to  have  been  defined  using  an  item  tagged  with  PP.  However,  if  boundary 
conditions  involving  particles  are  to  be  set,  the  second  label  should  point  to  an 
item  tagged  with  PP  that  defined  the  patch  particle  physics. 

4.2.2  OR 

The  tag  OR  is  followed  by  a  list  of  block  labels  in  the  order  of  assembly,  so  that 
e.g. 

ORyblZyblCybla 

indicates  that  information  associated  with  block  biz  supersedes  information  as¬ 
sociated  with  block  blc  which  in  turn  supersedes  information  about  block  bla. 
Equivalently  think  of  putting  biz  on  the  workbench,  then  attaching  blc  and  then 
bla. 


4.2.3  PS 

A  Plot  Set  is  specified  by  a  PS  tag  followed  by  a  file  format  name  -  tsd  for 
time  series  data,  grl  for  line  plots,  gst  for  particle  scatter  plots,  and  pnt,  lin 
and  pts  to  save  time  series,  line  plot  and  scatter  plot  data  respectively  for 
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subsequent  processing.  There  then  follows  the  name  of  a  Field  Set,  a  Domain 
Set  and  a  Time  Set.  The  format  is 

PSuf ile-f ormatuField-SetuDomain-SetuTime-Set . 


4.2.4  FS 

The  Field  Set  specifies  a  list  of  fields 

FSyf  ieldset-ncuneuf  ield-namelyf  ield-naine2uf  ield-name3u . 

The  names  of  fields  that  are  allowed  are: 

1.  El ,  E2,  E3,  HI ,  H2  and  H3  representing  the  components  of  the  electro¬ 
magnetic  fields  E  and  H  in  the  local  block  co-ordinate  system. 

2.  intE.dl  and  intH.dl  representing  the  line  integrals  /E-dl  and  /H-dl 
respectively.  Note  there  are  no  spaces  following  the  string  int. 

3.  into . dS,  intB . dS  and  int  j  .  dS  representing  the  surface  integrals  /  D.dS, 
/B.dS  and  / j.dS  respectively.  Note  there  are  no  spaces  following  the 
string  int. 

4.  intE .  D/2dV,  intB .  H/2dV  and  int  j  .  EdV  representing  the  volume  integrals 
of  |E  •  D,  •  H  and  J  ■  E  respectively.  Note  there  are  no  spaces  in  any 
of  these  labels. 

5.  Y(X)  where  X  and  Y  are  global  coordinates  for  2-D  scatterplots  as  follows: 
X  -  X  coordinate,  y  -  y  coordinate,  z  -  z  coordinate,  r  -  cylindrical 
polar  r  coordinate,  theta  -  cylindrical  polar  6  coordinate,  pi  -  physical 
component  of  momentum  in  the  1  mesh  direction,  p2  -  physical  compo¬ 
nent  of  mc>r:iientum  in  the  2  mesh  direction,  and  p3  -  physical  component 
of  momentum  in  the  3  mesh  direction. 

6.  a  label  appearing  in  a  line  containing  the  FL  tag. 

4.2.5  DO 

The  DOmain  specifies  a  list  of  subdomains 

DOudomain-nameuSubdomain-name luSubdomain-name2u . 

Note  that  fields  are  accumulated  over  elements  of  a  domain.  Primary  sub- 
domain  order  is  important,  and  in  particular  the  first  sub-domain  must  be  a 
primary. 


4.2.6  DS 


The  Domain  Set  specifies  a  list  of  domains  or  subdomains 
DSudomain-set-nameudomainluSubdomaiiilu . 
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4.2.7  SD 

The  tag  SD  defines  the  subdomains  which  are  secondary  to  the  first  listed 
subdomain  within  the  domain  domain-name. 

SDudomain-nameuprimary-subdomainusecondary-subdomainu . 

A  maximum  of  30  secondary  subdomains  are  allowed 

4.2.8  SG 

SG  specifies  a  signal  as  a  list  of  components 
SGusignal-nameucptlucpt2u. . . . 

Recognised  names  of  signals  are  boundary  for  setting  the  time  varying  ampli¬ 
tude  of  electromagnetic  boundary  conditions,  current  and  rf  current  similarly 
for  electric  current  forcing,  and  inject  similarly  for  injected  particle  beams. 


5  CLASS  DESCRIPTIONS 

5.1  Block  Geometry 

5.1.1  Rectbk 

This  is  the  name  of  the  base  subroutine  used  to  mesh  orthogonal  geometries, 
that  have  either  a  Cartesian  reference  coordinate  system  (MCTYPE  =  1),  which 
is  the  default  case,  or  a  cylindrical  polar  reference  coordinate  system  (MCTYPE 
=  2).  Twelve  edge  curves  defined  by  discrete  points  along  their  length  are 
required  to  define  the  block.  MTYPE  is  zero  for  uniform  spacing  of  the  grid  in 
curvilinear  coordinates.  The  variables  in  NAMELIST  RECTBK  are 


EDGES(MDEDGE) 

MCTYPE 

MDEDGE 

MTYPE(3) 


array  of  edge  curves 
reference  coordinate  type 
dimension  of  EDGES  array 
input  meshtype  selector 


5.1.2  Genblk 

This  is  the  name  of  the  base  subroutine  used  to  mesh  nonorthogonal  geometries, 
that  have  either  a  Cartesian  reference  coordinate  system  (MCTYPE  =  1),  which 
is  the  default  case,  or  a  cylindrical  polar  reference  coordinate  system  (MCTYPE 
=  2).  Twelve  edge  curves  defined  by  discrete  points  along  their  length  are 
required  to  define  the  block.  MTYPE  is  zero  for  uniform  spacing  of  the  grid  in 
curvilinear  coordinates.  The  variables  in  NAMELIST  GENBLK  are 

EDGES(MDEDGE)  array  of  edge  curves 
MCTYPE  reference  coordinate  type 

MDEDGE  dimension  of  EDGES  array 

MTYPE(3)  input  meshtype  selector 
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5.1.3  Parallelepiped 

A  parallelepiped  is  specified  by  the  lengths  of  its  three  sides  and  the  (spherical) 
polar  angles  of  two  of  them,  the  third  side  being  assumed  to  lie  along  the  polar 
axis  z.  The  variables  in  NAMELIST  PIPED  are 

RPHIl  azimuthal  angle  of  parallelepiped  side  1  (degrees) 

RPH12  azimuthal  angle  of  parallelepiped  side  2  (degrees) 

RSIDl  length  of  parallelepiped  side  1 

RSID2  length  of  parallelepiped  side  2 

RSID3  length  of  parallelepiped  side  3 

THETl  polar  angle  of  parallelepiped  side  1  (degrees) 

THET2  polar  angle  of  parallelepiped  side  2  (degrees) 


5.1.4  Polar  with  regular  meshing 

This  geometry  uses  a  cylindrical  polar  reference  coordinate  system.  The  vari¬ 
ables  in  NAMELIST  POLREG  are 


AXMAX 

maximum  along  cyl  polar-axis 

RADINR 

inner  radius 

RADOUT 

outer  radius 

THEMAX 

maximum  theta  (degrees) 

5.1.5  Variable  spaced  polar  lattice 

This  geometry  uses  a  cylindrical  polar  reference  coordinate  system.  The  vari¬ 
ables  in  NAMELIST  POLVAR  axe 

AXMAX  maximum  along  cyl  polar-axis 

RADINR  inner  radius 

RADOUT  outer  radius 

THEMAX  maximum  theta  (degrees) 


5.1.6  Variable  spaced  cubic  lattice 
The  variables  in  NAMELIST  CUBVAR  are 

RXMAX  maximum  along  x-axis 

RYMAX  maximum  along  y-axis 

RZMAX  maximum  along  z-axis 

5.1.7  Polar  mesh  segment 

The  variables  in  NAMELIST  POLSEG  are 
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AXMIN  minimum  along  cyl  polar-axis 

AX  MAX  maximum  along  cyl  polar-axis 

RADI  NR  inner  radius 

RADOUT  outer  radius 

THEMIN  minimum  theta  (degrees) 

THEMAX  maximum  theta  (degrees) 

5.1.8  Polar  to  rectangular  transition 

The  variables  in  NAMELIST  POLRCT  are 

AXMIN  minimum  along  cyl  polar-axis 

AXMAX  maximum  along  cyl  polar-axis 

RAD  CUR  radius  of  curved  edge 

RADSTR  radius  of  straight  edge 

THEMIN  minimum  theta  (degrees) 

THEMAX  maximum  theta  (degrees) 

5.1.9  Regular  cubic  lattice 

The  variables  in  NAMELIST  CUBREG  are 

RXMAX  maximum  along  x-axis 

RYMAX  maximum  along  y-axis 

RZMAX  maximum  along  z-axis 


5.1.10  Quadrilateral  cylinder 

This  geometry  is  the  extrusion  of  a  quadrilateral  by  a  length  RZMAX.  In  a  plane 
2  =  constant,  the  corners  of  the  quadrilateral  are  in  clockwise  order;  (0,0), 
(a;l,yl),  (x2,y2)  and  [RXMAX, RYMAX),  The  variables  in  NAMELIST 
QUADRI  are 

RXMAX  maximum  along  x-axis 

RYMAX  maximum  along  y-axis 

RZMAX  maximum  along  z-axis 

XQUADl  xl  point  for  quadrilateral  cylinder 

XQUAD2  x2  point  for  quadrilateral  cylinder 

YQUADl  yl  point  for  quadrilateral  cylinder 

YQUAD2  y2  point  for  quadrilateral  cylinder 

5.2  Block  Physics 
5.2.1  Uniform 

The  variables  in  NAMELIST  UNIFRM  are 
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EPSR  relative  permittivity  in  uniform  block 

RMUR  relative  permeability  in  uniform  block 


5.3  Particle  Properties 
5.3.1  Extra  and  Below 

These  classes  share  the  same  NAMELIST.  Class  extra  is  provided  so  that  just 
two  parameters,  controlling  the  particle  pushing  algorithm  may  be  set.  The  first 
parameter  NPBGEO,  when  set  to  unity,  forces  the  particle  patch  buffers  to  have 
the  format  which  is  consistent  with  the  particle  pushing  in  general  geometries 
which  have  an  underlying  cartesian  coordinate  system,  otherwise  it  is  assumed 
that  all  blocks  have  an  underlying  cylindrical  coordinate  system.  The  second  of 
these,  MALG  has  a  more  general  role,  but  the  array  entries  relevant  to  particle 
motion  are  as  follows. 

2  number  of  corrector  steps,  except  in  cylindrical  geometry  (see  MALG(18) 

below) . 

3  number  of  backward  steps  taken  to  try  to  get  the  “stagger”  right  on  the  first 

point  of  a  particle  orbit. 

16  if  set  to  unity,  use  a  stepwise  approximation  to  h2  in  cylindrical  geometry. 

18  if  greater  than  unity,  sets  the  number  of  corrector  steps  in  cylindrical  ge¬ 

ometry. 

19  if  set  to  unity,  do  not  update  the  electromagnetic  field  (useful  for  orbit 

calculations). 

20  if  set  to  unity,  suppress  header  output  in  gst  and  pts  plot  sets  (useful  if 

such  information  is  to  be  combined  to  represent  an  orbit). 

When  the  class  label  is  set  to  below,  a  simulation  will  BEgin  with  a  LOW 
number  of  particles  in  one  named  block  of  the  domain,  typically  for  orbit  tests 
or  to  stimulate  multipaction  effects.  Positive  NRAIMDP  is  the  number  of  par¬ 
ticles  introduced  with  uniformly  randomly  distributed  velocities  subject  to  a 
maximum  speed  SPEEDM  given  in  units  of  c  =  3  x  10®m/s.  If  NRANDP  is  zero, 
the  user  may  specify  each  particle  separately  The  number  of  particles  is  then 
set  implicitly  -  a  negative  first  positional  coordinate  UCPOS  implies  the  end  of 
the  particle  list.  The  variables  in  NAMELIST  POLE  are 

CBLOCK*32  block  label 

MALG(20)  algorithm  parameters 

NOVEL  coordinate  system  for  pcle  vel 

NPBGEO  particle  buffer  geometry 

NRANDP  number  of  particles  in  random  start 

NODIM  number  of  position  coordinates 

SPEEDM  max  speed  of  random  start  particles 
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UCPOS(3,100)  particle  position  (unit  cube) 
VEL(3,100)  particle  velocity  (m/s  global) 


5.4  Patch  Physics 

5.4.1  Perfect  conductor,  Polar  2Lx;is  and  Reflector 

These  classes  have  no  need  for  an  associated  namelist. 


5.4.2  Applied  field,  Resistive  wall  and  Surface  field 

These  classes  share  the  same  NAMELIST.  In  the  case  of  an  applied  electric 
displacement,  the  components  of  field  are  relative  to  the  coordinates  of  the 
block;  note  that  only  in  some  circumstances  is  d  constant.  STHETA  corresponds 
to  0  in  Ref  [7].  Surface  field  implies  a  boundary  condition  of  resistive  wall  plus 
applied  field.  The  variables  in  NAMELIST  BCS  are 


DAPLYA(3) 

DPOT 

EAMP 

NGLOBC 

SF0RM*8 

SPOWER 

STHETA 

SURFZ 


applied  electric  field  marker 

potential  difference  in  V  across  block  (DAPLYA  marks  direction) 

electric  field  amplitude  in  Vm~^ 

if  1  use  ic  as  be,  if  0  use  applied  field 

description  of  signal 

signal  power 

lagging  factor  for  resistive  wall  be 
surface  impedance 


DAPLYA  sets  the  phase  shifts  RPHSHF  in  the  ic  used  as  be  case.  It  should  be 
a  vector  normal  to  the  boundary  so  that  a  null  of  electric  field  d  is  avoided. 


5.4.3  Emitter,  Inject  beam  and  Multipaction 

These  classes  all  use  the  same  NAMELIST  to  specify  particle  emission  on  a 
patch.  The  relevant  variables  for  space  charge  limited  emission  are  ALFA, 
CSPECI,  EFFE  and  NEMALG,  whilst  those  characterising  beamed  emission  are 
CSPECI,  CURBEA,  RADBEA,  ROTBEA  and  VBEAM.  The  multipaction  or  sec¬ 
ondary  emission  parameters  are  defined  in  Ref  [8].  The  variables  in  NAMELIST 
PARBCS  are 


ALFA 

CSPECI*32 

CURBEA 

DELTAM 

EFFE 

EMCRIT 

EMEl 

EME2 

EMMAX 


emission  splitting  parameter 
species  name 
beam  current 

multipaction  parameter  5max 
effective  E  factor 

critical  energy  for  multipaction  emission 
multipaction  parameter  E\ 
multipaction  parameter  E2 
multipaction  parameter  Emax 
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EMMEAN 

mean  energy  of  inelastic  emission 

EMSTDD 

std  dev.  energy  of  inelastic  emission 

NEMALG 

emission  algorithm  (2  for  2-D) 

NERAND 

random  emission  and/or  specular  reflection 

NHYPER 

multipaction  splitting  parameter 

NLSEDA 

true  for  use  of  data  array 

NLSEE 

true  for  variable  emission  coefficient 

RADBEA(2) 

beam  radii  (inner  then  outer) 

ROTBEA 

beam  rotation 

SIGMES(200) 

multipaction  parameter  array 

VBEAM 

beam  voltage 

5.5  Other  Geometry 
5.5.1  Inherit 


This  defines  a  geometrical  object  by  inheritance,  ie  as  parts  of  predefined  ob¬ 
jects.  Normally  the  object  is  also  defined  by  extension,  so  that  an  object  that 
crosses  many  block  boundaries  can  be  defined  from  say  a  line  in  one  given  block. 
The  relevant  variables  are  set  using  NAMELIST  INHERI  which  contains 


CBLOCK*32 

CUNIVS(512)*32 

NBRANC(2) 

NDIR(3) 

NLBOTH(3) 

NLOXS 

NLXTEN 

OPOINT(3) 

OUNIVS(3,512) 

XP01NT(3) 

XUN1VS(3,512) 


block  label 

block  labels  of  universe 

branching  directions  (defines  paths) 

direction  markers 

true  for  both  ways  in  direction 

true  if  o  and  x  points  in  universe 

true  to  extend  geometry 

o-point  real  coords 

o-points  in  universe 

x-point  real  coords 

x-points  in  universe 


5.6  Miscellaneous 
5.6.1  Free  parameters 

This  provides  the  opportunity  to  designate  a  specific  block  to  provide  the  ori¬ 
gin  of  coordinates  (using  those  variables  ending  in  GLB),  to  set  parameters  to 
control  a  run  (IMRUN,  MDPART,  NOPSEL  and  IMXPTDD),  and  to  control  diag¬ 
nostic  output  as  described  in  Ref  [6].  MDPART  =  1  prevents  the  introduction 
of  particles,  but  both  it  or  NBEXT  will  be  overridden  by  any  use  of  respectively 
the  SP  or  PO  tags.  The  value  of  NXPTDD  has  the  following  effect: 

1.  print  EHWSP  before  and  after  patch  exchange 

2.  print  D  and  EHWSP  before  and  after  2nd  patch  exchange 
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3.  particle  diagnostics  including  positions 

4.  print  B,  D  and  EHWSP 

5.  use  with  debugger 

6.  dump  EHWSP  on  every  call  of  subroutine  EXPT 

7.  print  B  and  D 

8.  dump  C  on  every  call  of  subroutine  EXPT 

9.  print  total  number  of  particles  in  each  block  every  step 

DTMUL  is  the  Courant  number,  which  multiplies  the  right-hand  side  of  equation 
(38)  of  Ref  [9]. 

The  variables  in  NAMELIST  SF  which  have  not  already  been  described  in 
Ref  [6]  or  in  the  section  for  tag  PO,  are 


CBLGLB*32 

DTMUL 

MALG 

MDPART 

NBEXT 

NLDUMP 

NOPSEL 

NPAR(MAXPAR) 

NRUN 

NTPRES 

RFIGLB 

RPAR(MAXPAR) 

THEGLB 

XYZGLB(3) 


block  with  designated  origin  and  rotation 
timestep  multiplier 

if  entry  17  is  unity,  discrete  correction  to  ic 

Dimension  of  PARTicle  coordinate  arrays 

use  initial  B  as  external  B 

true  for  binary  dump 

Select  Output  Sequence 

integer  parameter  defaults  on  tagged  line 

number  of  time  steps 

number  of  processes  needed  on  target 

spherical  phi  rotn  of  named  block 

real  parameter  defaults  on  tagged  line 

spherical  theta  rotn  of  named  block 

physical  position  of  named  block 
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Abstract 

The  MPICTIM  code  is  designed  to  examine  time  series  data  produced 
by  PIC  simulation  codes.  It  uses  an  OSF/Motif  [1]  Graphical  User  In¬ 
terface  (GUI)  instead  of  the  character  based  interface;  this  OSF/Motif 
interface  to  MPICTIM  is  much  easier  than  a  Command  Line  Interface 
(CLI)  for  interactive  use. 


1  New  Features  in  MPICTIM  Version  2.10.00 

This  User’s  Guide  describes  MPICTIM  Version  2.10.00  which  has  replaced 
MPICTIM  Version  1.00.00  which  was  described  in  [2]. 

The  following  new  features  have  been  added  to  MPICTIM: 

1.  Values  can  be  extracted  from  the  plot  by  using  the  Get  Values  button 
and  clicking  on  the  plot  with  the  left  mouse  button. 

2.  Plots  using  logarithmic  scales  can  be  produced  by  activating  the  Log 
toggle  button. 

3.  Graticules  can  be  produced  by  activating  the  Graticule  toggle  button. 


2  Using  MPICTIM 

This  Section  contains  a  task  orientated  description  of  how  to  use  MPICTIM. 
Section  3  describes  the  functions  of  the  different  components  of  MPICTIM.  The 
MPICTIM  application  uses  OSF/Motif  Graphical  User  Interface  components. 
You  should  refer  to  your  system’s  documentation  for  general  information  on 
using  OSF/Motif.  This  document  concentrates  on  aspects  which  are  specific  to 
MPICTIM. 

2.1  Starting  MPICTIM 

To  start  MPICTIM  start  an  xterm  window  and  issue  the  mpictim  command  at 
the  prompt: 

$  mpictim 
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Figure  1;  Motif  Time  Series  Analysis  Program  MPICTIM 
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Move  and  resize  the  xterm  window  to  the  lower  right  of  the  screen  as  shown 
in  Figure  1.  Your  workstation  screen  should  now  look  like  the  screen  shown  in 
Figure  1.  You  should  consult  your  system  administrator  if  it  looks  substantially 
different. 

2.2  Quitting  MPICTIM 

To  quit  the  application  activate  the  File  pull  down  menu  from  the  menu  bar 
(Figure  2)  and  select  the  Exit  option. 

2.3  Opening  Data  Files 

To  open  a  time  series  data  file  activate  the  File  pull  down  menu  from  the  menu 
bar  (Figure  2)  and  select  the  Open  option  to  pop  up  the  Open  File  dialogue 
window  (Figure  3).  Use  the  Open  File  dialogue  window  load  a  file  by  selecting 
a  file  and  activating  the  OK  button.  Opening  a  time  series  data  file  replaces 
all  time  series  data  held  in  the  code.  If  you  have  sequential  data  from  restart 
calculations  which  occupies  several  files  use  should  use  the  Append  option  from 
the  File  menu  to  append  data  without  deleting  data  which  is  already  loaded. 


Figure  2:  File  Menu 


2.4  Setting  Processing  Options 

The  data  processing  options  are  set  in  the  Processing  Options  area.  The  options 
take  effect  when  a  data  item  is  selected  and  either  of  the  Read  Audit  Data  or 
Read  Surface  Data  buttons  are  activated. 

The  period  used  by  the  mean,  rms,  variance  and  spectrum  plots  is  set  by 
typing  text  in  the  Period  text  field  and  activating  the  Apply  button  at  the 
bottom  of  the  main  window. 

The  Diff  toggle  button  enables  differentiation  of  the  data  before  other  pro¬ 
cessing  is  performed. 

The  Processing  option  menu  (Figure  4)  allows  the  user  to  choose  how  the 
data  is  processed. 

The  Int  toggle  button  enables  integration  of  the  data  after  other  processing 
has  been  performed. 
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Open  File.popup 


Filter 

/hoine/nick/3DPIC/PIC3D/0UT/*,ts4 


Directories  Files 


Lhome/n  i  ck/SDP  I  C/P  I C3D/CIIJT/ .  H 

1 

3 

/home/nick/3DPIC/PlC3D/0UT/.. 

sbcl6z-02,tsd 

sbclGzn-Ol.tsd 

sbclBzn-02,tsd 

QHHHEIIOI 

Selection 

/'hone/nick/3DPIC/PIC3D/0UT/sbclB2-01,tscI 


OK 

i 

! 

Filter 

Cancel 

Figure  3:  File  Selection  Dialogue 
The  Log  toggle  button  enables  logarithmic  scale  plots. 
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Figure  4;  Data  Menu 


2.5  Choosing  the  Data  to  be  Processed 

To  choose  the  data  to  be  processed  select  an  audit  variable  from  the  Audit  Data 
scrolling  list  or  select  select  a  surface  and  a  surface  variable  from  the  Surfaces 
and  Surface  Data  scrolling  lists. 

Activate  the  Read  Audit  Data  button  or  the  Read  Surface  Data  button  to 
process  the  data. 
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2.6  Producing  Plots 

The  user  may  select  the  colour  used  for  plots  using  the  Colour  option  menu 
(Figure  5).  The  plot  label  is  set  in  the  Label  next  to  the  Plot  button.  The  user 
should  set  the  ranges  for  the  plots  in  the  ranges  fields.  The  Apply  button  at  the 
bottom  of  the  main  window  must  be  activated  to  make  the  values  active.  The 
time  range  is  also  used  as  a  window  when  processing  data.  The  Reset,  Suggest 
and  Data  buttons  at  the  bottom  of  the  main  window  may  be  used  to  display 
current  ranges,  suggested  ranges  and  data  ranges  (and  other  parameters). 

Once  the  parameters  are  set  activating  the  Plot  button  displays  the  plot  on 
the  screen. 


Figure  5:  Colour  Menu 


2.7  Annotating  Plots 

The  labels  used  for  annotating  the  plot  are  set  in  the  labels  text  fields.  The 
Apply  button  at  the  bottom  of  the  main  window  must  be  activated  to  make  the 
labels  active.  The  Reset,  Suggest  and  Data  buttons  at  the  bottom  of  the  main 
window  may  be  used  to  display  current  labels,  suggested  labels  and  labels  from 
the  data  file  (and  other  parameters).  The  Graticule  toggle  button  enables  the 
plotting  of  graticules. 

Once  the  labels  are  set  activating  the  Annotate  button  annotates  the  plot 
on  the  screen  and  plots  the  axes  and  graticule  (if  enabled). 


2.8  Saving  and  Erasing  Plots 

To  save  a  plot  activate  the  Save  button.  The  plot  is  saved  to  a  GHOST  GRID 
file  [3]  with  the  name  given  in  the  text  field  immediately  to  the  right  of  the  save 
button. 

To  erase  a  plot  activate  the  Erase  button. 
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2.9  Viewing  Parameters 


Information  from  the  time  series  data  files  can  be  displayed  in  the  window  from 
which  MPICTIM  was  started  by  activating  one  of  the  options  from  the  View 
pull  down  menu  on  the  menu  bar  (Figure  6). 


File 


Plot 


Period 


View 

Help 

Audit  Data  | 

— 

Surface  Data 

surfaces 

— 

Files 

. 

Audit  Data 


Figure  6:  View  Menu 


2.10  Extracting  Data  from  the  Plot 

To  extract  data  from  the  plot  activate  the  Get  Values  button  at  the  bottom  of 
the  main  window.  The  controls  will  then  be  grayed  out  to  indicate  that  they 
are  inactive  and  a  message  similar  to  the  following  will  be  printed  in  the  xterm 
window: 

Collecting  values  from  graph: 

click  mouse  left  button  for  data,  right  for  done 

x:  Time  ps 

y:  El  none 

The  text  after  the  x:  and  y:  gives  the  name  and  units  of  the  current 
variables  being  plotted. 

Each  time  the  left  mouse  button  is  clicked  at  a  point  on  the  plot  the  coor¬ 
dinates  are  printed  in  the  xterm  window  using  the  current  scales:  E.g. 


X  = 

192.71 

y  = 

9.9188 

X  = 

249.40 

y  = 

-1.9838 

X  = 

370.25 

y  = 

-77.570 

When  the  right  mouse  button  is  clicked  on  the  plot  the  controls  on  the  main 
window  become  activate  and  are  ungrayed  and  the  following  message  is  printed 
in  the  xterm  window 


Data  collection  finished 
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3  Components  of  MPICTIM 

This  Section  describes  the  functions  of  the  different  components  of  MPICTIM. 
Section  2  contains  a  task  orientated  description  of  how  to  use  MPICTIM. 

Figure  1  shows  the  two  base  windows  of  the  MPICTIM  application.  The 
main  control  window  is  top  right  and  the  graphics  display  window  is  top  left. 
The  terminal  window  from  which  MPICTIM  was  started  is  bottom  right. 

The  main  control  area  is  shown  in  greater  detail  in  Figure  7.  The  compo¬ 
nents  of  this  window  are  shown  in  greater  detail  in  Figure  7  and  are  described 
in  the  following  subsections.  The  components  are  described  in  order  starting 
from  the  top  left  and  scanning  from  left  to  right  and  from  top  to  bottom. 


Figure  7:  Main  Window 
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3.1  Menu  Bar 

The  menu  bar  contains  two  pull  down  menus:  the  File  menu  and  the  View 

menu. 

3.1.1  File  Menu 

The  File  menu  contains  three  items: 

Open  pops  up  a  file  selection  dialogue  to  load  data  from  a  time  series  data  file. 
Loading  the  data  file  replaces  any  data  already  loaded. 

Append  pops  up  a  file  selection  dialogue  to  append  data  from  a  time  series 
data  file  to  data  already  loaded.  The  time  series  data  file  must  contain 
the  same  data  items  and  use  the  same  time  sampling  interval  as  the  files 
which  are  already  loaded. 

Exit  quits  the  application. 


3.1.2  View  Menu 

The  View  menu  is  used  for  displaying  information  in  the  window  from  which 
MPICTIM  was  started.  The  View  menu  contains  four  items: 

Audit  Data  displays  a  list  of  audit  variables.  The  list  gives  the  variable  num¬ 
bers,  the  labels,  the  units  used  and  the  long  names. 

Surface  Data  displays  a  list  of  surface  variables.  The  list  gives  the  variable 
numbers,  the  labels,  the  units  used  and  the  long  names. 

surfaces  displays  a  list  of  surfaces.  The  list  gives  the  variable  numbers,  the 
labels,  the  default  plot  colours  and  the  long  names. 

Files  displays  a  list  of  the  files  that  have  been  loaded.  The  list  gives  the  file 
name,  the  run  identification  string,  the  sample  range  in  steps,  the  sample 
range  in  units  of  time,  the  hinted  period  in  steps  and  the  sampling  interval. 

3.2  Plot  Controls 

The  plot  controls  region  contains  controls  used  to  plot  processed  data.  Working 
from  left  to  right  the  controls  in  this  area  are: 

3.2.1  Plot  Button 

Pressing  the  plot  button  plots  the  processed  data  on  the  screen  using  the  applied 
ranges.  The  plot  is  labelled  with  the  text  from  the  Label  field  (immediately  to 
the  left  of  the  plot  button)  and  is  plotted  in  the  colour  set  in  the  Colour  option 


menu. 
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3.2.2  Label  Field 

The  label  field  is  immediately  to  the  left  of  the  plot  button.  The  text  in  this 
field  (up  to  8  characters)  is  used  as  the  label  for  the  current  curve  being  plotted. 
This  field  may  can  be  made  blank  if  no  label  is  required. 

3.2.3  Colour  Option  Menu 

This  control  sets  the  colour  used  to  plot  a  curve.  The  colour  choices  are 
black, red,  green,  blue,  white,  cyan,  magenta,  yellow.  (White  will  not  be  visible 
because  the  background  is  white). 

3.2.4  Annotate  Button 

This  button  annotates  the  plot  with  the  applied  labels  from  the  Labels  Area. 
Two  title  lines  and  labels  for  the  axes  are  added. 


3.2.5  Erase  Button 

This  button  erases  the  plot. 


3.2.6  Save  Button 

This  button  saves  the  plot  to  a  file  with  the  name  given  in  the  file  name  field 
immediately  to  the  left  and  erase  the  plot. 

3.2.7  File  Name  Field 

The  file  name  field  is  immediately  to  the  left  of  the  Save  button.  It  may  contain 
up  to  16  characters  of  text.  It  specifies  the  name  of  the  GHOST  grid  file  in 
which  the  current  plot  will  be  stored.  The  file  is  placed  in  the  directory  in  which 
MPICTIM  was  started. 

3.3  Processing  Controls 

This  area  contains  controls  which  determine  how  a  data  item  is  processed  when 
it  is  read. 

3.3.1  Period  Field 

This  field  is  used  for  setting  and  displaying  the  period  used  by  the  mean,  rms, 
variance  and  spectrum  processing  options.  The  period  is  set  when  the  Apply 
button  in  the  Buttons  area  at  the  bottom  of  the  application  is  activated.  The 
hinted  period  fi:om  the  data  file  is  shown  when  the  Data  button  is  activated, 
the  suggested  period  is  shown  when  the  Suggest  button  is  activated  and  the 
current  value  is  when  the  Reset  button  is  activated. 
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3.3.2  DifF  Toggle  Button 

When  the  DifF  Toggle  button  is  active  the  data  will  be  differentiated  with  respect 
to  time  before  other  processing  takes  place. 


3.3.3  Processing  Option  Menu 

This  option  menu  selects  the  type  of  processing  applied  to  the  data  when  it  is 
processed.  There  are  six  options  available: 

data  is  unchanged 

mean  computed  over  current  period 

rms  (root  mean  squared)  values  computed  over  current  period 
variance  computed  of  current  period 

spectrum  computed  for  time  interval  truncated  to  an  integral  multiple  of  the 
current  period 

strobe  plot  computed  for  time  interval  truncated  to  an  integral  multiple  of  the 
hinted  period 


3.3.4  Int  Toggle  Button 

When  the  Int  Toggle  button  is  active  the  data  will  be  integrated  with  respect 
to  the  independent  variable  after  other  processing  has  taken  place. 

3.3.5  Graticule  Toggle  Button 

When  the  Graticule  Toggle  button  is  active  a  graticule  will  be  plotted  when 
the  Annotate  button  is  activated. 

3.3.6  Log  Toggle  Button 

When  the  Log  Toggle  button  is  active  the  data  will  be  plotted  using  logarithmic 
scales  when  the  Plot  button  is  activated. 

3.4  Audit  Data  Selection  Area 

This  area  is  used  to  select  an  Audit  Data  item  and  process  it  using  the  current 
options.  The  user  selects  the  variable  in  the  scrolling  list,  the  long  label  is  then 
shown  in  the  selection  text  field  and  the  data  is  processed  by  activating  the 
button  labelled  “Read  Audit  Data”.  The  processed  data  remains  unchanged 
until  another  Audit  Data  item  or  a  Surface  Data  item  is  read. 
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3.5  Surface  Data  Selection  Area 

This  area  is  used  to  select  an  Surface  Data  item  and  process  it  using  the  current 
options.  The  user  selects  the  surface  and  the  variable  in  the  scrolling  lists,  the 
long  labels  are  then  shown  in  the  selection  text  fields  and  the  data  is  processed 
by  activating  the  button  labelled  “Read  Surface  Data”.  The  processed  data 
remains  unchanged  until  another  a  Surface  Data  item  or  Audit  Data  item  is 
read. 


3.6  Ranges  Area 

This  area  contains  text  fields  for  viewing  and  setting  various  ranges.  The  values 
in  the  text  fields  are  used  to  set  the  ranges  when  the  Apply  button  in  the 
Buttons  area  at  the  bottom  of  the  window  is  activated.  The  text  fields  are 
set  to  the  current  values  when  the  Reset  button  is  activated,  they  are  set  to 
suggested  values  when  the  Suggest  button  is  activated  and  are  set  to  the  data 
ranges  (after  processing)  when  the  Data  button  is  activated. 


3.6.1  Time  Range 

These  text  fields  may  show  the  time  range  of  the  processed  data,  the  suggested 
time  range  of  the  processed  data,  the  current  time  range  or  a  range  which 
may  be  applied,  depending  on  which  of  the  Apply,  Reset,  Suggest  or  Data 
buttons  have  been  activated.  This  interval  is  used  as  a  data  window  during 
the  processing  of  data  i.e.  data  outside  the  interval  is  discarded.  This  interval 
is  also  used  as  the  time  interval  for  plots  which  have  time  as  the  independent 
variable. 


3.6.2  Frequency  Range 

These  text  fields  may  show  the  frequency  range  of  the  processed  data,  the 
suggested  frequency  range  of  the  processed  data,  the  current  frequency  range 
or  a  range  which  may  be  applied,  depending  on  which  of  the  Apply,  Reset, 
Suggest  or  Data  buttons  have  been  activated.  This  interval  is  used  as  the 
frequency  interval  for  plots  which  have  frequency  as  the  independent  variable. 
It  is  also  used  as  the  interval  for  the  strobe  plots  in  which  data  from  an  interval 
is  processed  to  produce  one  period  of  the  signal  as  a  function  of  time.  (For 
Strobe  plots  the  time  axis  is  in  units  of  the  hinted  period). 


3.6.3  Variable  Range 

These  text  fields  may  show  the  dependent  variable  range  of  the  processed  data, 
the  suggested  dependent  variable  range  of  the  processed  data,  the  current  de¬ 
pendent  variable  range  or  a  range  which  may  be  applied,  depending  on  which  of 
the  Apply,  Reset,  Suggest  or  Data  buttons  have  been  activated.  This  interval 
is  used  as  the  dependent  variable  interval  for  all  plots. 
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3.7  Labels  Region 

This  area  contains  text  fields  for  viewing  and  setting  various  labels  used  for 
annotating  the  plots.  The  values  in  the  text  fields  are  used  to  set  the  labels 
when  the  Apply  button  in  the  Buttons  area  at  the  bottom  of  the  window  is 
activated.  The  text  fields  are  set  to  the  current  values  when  the  Reset  button  is 
activated,  they  are  set  to  suggested  values  when  the  Suggest  button  is  activated 
and  values  from  the  data  file  when  the  Data  button  is  activated. 

Two  lines  of  titles  and  labels  for  the  x  and  y  axes  may  be  set  and  viewed. 

3.8  Buttons  Area 

This  area  contains  four  buttons  which  control  parameters  used  by  the  applica¬ 
tion: 

Apply  This  button  takes  the  parameters  from  the  Ranges  area,  the  Labels 
area,  the  plot  label  field,  the  Colour  option  menu  and  the  Period  field  and 
makes  the  current. 

Reset  This  button  resets  the  Ranges  area  text  fields,  the  Labels  area  text 
fields,  the  plot  label  field,  the  Colour  option  menu  and  the  Period  text 
field  to  the  current  values. 

Suggest  This  button  sets  the  Ranges  area  text  fields,  the  Labels  area  text 
fields,  the  plot  label  field,  the  Colour  option  menu  and  the  Period  text 
field  to  the  suggested  values. 

Data  This  button  sets  the  Ranges  area  text  fields,  the  Labels  area  text  fields, 
the  plot  label  field,  the  Colour  option  menu  and  the  Period  text  field  to 
the  actual  ranges  of  the  data  and  values  from  the  time  series  data  files. 

Get  Values  This  button  starts  data  collection  from  the  plot.  It  is  described 
in  Section  2.10. 

4  Installation  Notes 

4.1  Solaris  2.x  Distribution 

The  MPICTIM  2.10.00  distribution  for  Solaris  2.3  and  later  consists  of  the 
mpictim  executable,  the  Mpictim  application  defaults  file,  a  README  file  and 
example  resources  for  the  mwm  and  olwm  window  managers. 

The  Mpictim  application  defaults  file  should  normally  be  installed  in  the 
/usr/openwinhome/lib/app-defaults  directory  but  some  systems  may  use  a 
different  location. 

The  mpictim  executable  uses  GHOST,  Motif  and  Fortran  dynamically  loaded 
libraries  and  requires  the  GHOST  xgenie  executable  and  the  GHOST  fonts. 
The  GHOST  environment  is  available  from  AEA  Technology.  The  Motif  1.2.3 
dynamic  libraries  are  in  the  SUNWmfrun  package  from  the  Solaris  2.4  distribu¬ 
tion  (they  can  also  be  installed  on  Solaris  2.3).  The  Fortran  dynamic  libraries 
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are  in  the  SPR01ib77  package  which  is  distributed  with  SPARCompiler  Fortran 
by  SunSoft.  (SPARCompiler  Fortran  3.0.1  was  used  to  produce  the  code  but 
the  libraries  from  versions  2.0,  2.0.1  and  3.0  should  also  work).  If  the  GHOST, 
Motif  and  Fortran  are  installed  in  the  standard  locations  then  mpictim  should 
run  straight  away,  otherwise  environment  variables  will  need  to  be  set  to  make 
the  application  work. 
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Abstract 

This  paper  describes  the  algorithm  and  implementation  issues  for  the  three-dimensional  body-fitted  Particle-In-Cell  (PIC) 
software  suite  3DPIC  for  modelling  the  time  evolution  of  interactions  between  electromagnetic  (EM)  fields  and  flows  of 
relativistic  charged  particles.  3DPIC  is  applicable  to  EM  calculations  in  complex  geometries  where  displacement  fields  are 
important,  and  was  developed  specifically  for  microwave  device  modelling.  A  description  is  given  of  the  physical  model, 
the  numerical  scheme  and  the  software.  The  software  was  designed  to  run  efficiently  on  both  serial  and  distributed-memory 
parallel  computers.  The  performance  achieved  by  the  software  on'parallel  computers  demonstrates  the  potential  of  this  code 
for  large  scale  time-dependent  electromagnetic  and  electrodynamic  calculations. 


1.  Introduction 

The  software  described  in  this  paper  offers  pow¬ 
erful  new  capabilities  for  electromagnetic  PIC  [1,2] 
modelling.  It  was  designed  for  two-dimensional  (2-D) 
and  three-dimensional  (3-D)  modelling  of  microwave 
tubes  and  microwave  transmission  where  the  interac¬ 
tion  of  electromagnetic  waves  with  charged  particle 
flow  is  important  [4-6],  although  it  can  equally  well 
be  applied  to  other  time  dependent  problems  normally 
tackled  by  finite  difference,  time  domain  (FDTD) 
codes.  The  software  solves  the  relativistic  Maxwell- 
Vlasov  set  of  equations  for  the  electromagnetic  field 
and  particle  motion  throughout  the  computational  vol¬ 
ume;  this  is  appropriate  for  situations  where  the  char¬ 
acteristic  timescales  are  comparable  to  the  transit  time 
of  light  across  the  system.  For  phenomena  on  a  slower 
timescale,  electrostatic  [1]  or  Darwin  models  [3]  may 
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be  more  appropriate.  Volume  filling  lossy  dielectrics, 
magnetic  media,  conducting  grids  and  surfaces,  sur¬ 
face  physics  and  coupling  to  external  circuits  can  all 
be  incorporated  into  the  simulations. 

Particle-in-Cell  (PIC)  simulation  methods  have 
been  used  to  solve  the  coupled  Maxwell-Vlasov  sys¬ 
tem  of  equations  for  over  two  decades.  Early  simula¬ 
tions  (e.g.  [7] )  were  restricted  to  simple  geometries, 
and  repeatedly  solved  Poisson’s  equation  to  correct 
the  longitudinal  part  of  the  electric  field  to  obtain 
charge  conservation.  Later  codes,  designed  specif¬ 
ically  for  simulations  in  non-rectangular  domains 
achieved  charge  conservation  without  the  solution 
of  Poisson’s  equation  by  using  a  particle  integration 
scheme  which  moved  particles  along  the  principal 
axes  of  the  grid.  Mature  codes  using  this  approach  in¬ 
clude  theMRC  MAGIC  (2-D)  [8,9]  and  SOS  (3-D) 

[  10]  codes,  and  the  SAIC  3-D  code  ARGUS  [11]. 

MAGIC,  SOS  and  other  codes  of  similar  vintage 
were  designed  to  exploit  the  large  scientific  com- 
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puters  of  the  1970s.  More  recent  programs,  such 
as  QUICKSILVER  [12]  were  developed  to  exploit 
shared-memory  MIMD  computers  efficiently.  Almost 
all  of  the  mature  codes  use  finite  differences  and  are 
restricted  to  meshes  which  are  unions  of  rectangular 
bricks.  None  are  specifically  designed  to  use  body 
fitted  coordinates  and  to  exploit  distributed-memory 
MIMD  computers. 

The  new  simulation  suite  3DPIC  differs  from  es¬ 
tablished  PIC  codes  [8-12]  for  microwave  modelling 
primarily  in  that 

(i)  it  uses  body-fitted  finite  elements  rather  than  fi¬ 
nite  differences  on  an  orthogonal  grid, 

(ii)  the  finite  element  (FE)  derivation  carries  over 
the  simplicity  of  the  Yee  FDTD  algorithm  [  13] 
for  the  EM  calculation  to  general  geometry, 

(iii)  the  general  geometry  FE  derivation  gives  current 
assignment  schemes  which  are  charge  conserv¬ 
ing, 

(iv)  it  was  designed  ab  initio  to  use  distributed- 
memory  parallel  computers  [14]  efficiently. 

The  last  is  achieved  by  a  multiblock  decomposition  of 
the  computational  volume.  Each  block  in  the  decom¬ 
position  behaves  as  an  independent  PIC  calculation, 
which  communicates  with  neighbouring  blocks  by  ex¬ 
changing  fields  and  particles  at  its  boundaries;  these 
exchanges  are  treated  as  message  passing,  thus  allow¬ 
ing  simple  and  efficient  mapping  of  complex  physical 
configurations  onto  distributed-memory  parallel  com¬ 
puters. 

Blocks  are  curvilinear  hexahedra,  i.e.  they  are 
cuboid  in  some  (in  general  non-orthogonal)  coordi¬ 
nate  system.  The  elegance  of  this  method  is  that  with 
appropriate  choices  of  co-  and  contra-  variant  tensor 
components,  the  majority  of  the  PIC  algorithm  for 
each  block  takes  a  coordinate  invariant  form  which 
differs  little  from  the  “virtual  particle”  scheme  [15] 
on  regular  Cartesian  meshes.  All  geometrical  and  ma¬ 
terial  properties  are  encapsulated  in  the  constitutive 
equations  for  the  fields  and  in  coordinate  transforma¬ 
tions  in  the  particle  equations  of  motion. 

The  next  section  presents  the  physical  model,  fol¬ 
lowed  in  Section  3  by  an  outline  of  the  numerical 
method.  Our  FE  derivation  extends  the  methods  intro¬ 
duced  by  Lewis  [16],  in  two  directions:  the  employ¬ 
ment  of  more  general  element  shapes  and  the  use  of 
the  FE  approximation  for  time  dependence  in  the  field 
action  integral  [15].  The  former  allow  accurate  ele¬ 


ment  fitting  to  complex  shaped  devices,  and  the  latter 
automatically  ensures  charge  conservation.  The  im¬ 
portance  of  exact  charge  conservation  is  that  it  elim¬ 
inates  the  need  to  assign  charge  and  solve  Poisson’s 
equation  to  correct  the  longitudinal  electric  field.  Ma¬ 
jor  issues  are  the  description  of  objects  and  compact 
data  storage  for  the  computations;  these  issues  are  ad¬ 
dressed  in  the  Section  4.  Implementing  the  code  on 
parallel  computers  and  benchmarking  its  performance 
are  treated  in  Section  5.  Section  6  contains  concluding 
remarks. 


2.  Physical  model 

2. 7.  Basic  equations 

The  mathematical  model  consists  of  the  Maxwell- 
Vlasov  set;  Maxwell’s  equations  for  the  electromag¬ 
netic  fields  and  Vlasov’s  equation  (possibly  relativis¬ 
tic)  for  the  dynamics  of  the  charged  particles.  The 
charged  particles  may  be  electrons,  ions,  or  both,  al¬ 
though  the  discussion  given  below  is  for  electrons 
only.  The  mathematical  model  of  the  physical  sys¬ 
tem  is  completed  by  adding  descriptions  of  the  emis¬ 
sion  and  absorption  of  radiation  and  of  particles  from 
boundaries. 

If  the  particle  distribution  function  /  is  represented 
by  a  set  of  sample  points  (i.e.  “superparticles”) 

f{x,p,t)  =  Y^Ns8{x  -  Xi{t))d{p  ~  pM)  (1) 

i 

where  (x,,  p,  );  /e[  1,  T/p]  are  the  coordinates  of  the 
Np  superparticles,  each  of  mass  M  =  N^mo  and  charge 
Q  -  bfsq,  then  the  Maxwell-Vlasov  set  may  be  written 
in  terms  of  the  action  integral 

-yE— (2) 

where  symbols  take  their  usual  meanings;  A  is  the 
magnetic  vector  potential,  (ft  is  the  electric  potential, 
B  =  V  X  A  is  magnetic  flux,  £=-V0-Ais  elec¬ 
tric  field,  D  is  electric  displacement,  H  is  magnetic 
intensity,  p  is  charge  density,  and  j  is  current  density. 
y  is  the  relativistic  gamma,  and  the  integrals  are  over 
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time  1  and  space  r.  Treating  /  as  a  functional  of  the 
vector  potential  A,  the  scalar  potential  (f)  and  particle 
coordinates  {jr,}  leads  to  Euler-Lagrange  equations 
which  reduce  to  Maxwell’s  equations  and  the  relativis¬ 
tic  equations  of  motion  [16,17]. 

2.2.  Material  properties 

General  dielectric  and  magnetic  media  may  be  in¬ 
cluded  in  the  model  by  adding  polarisation,  P,  and 
magnetisation,  M,  terms  to  the  action  integral 

!  =  ■■■  + J  dtdT(M‘B^P-E)  (3) 

and  substituting  for  M  and  P  from  the  resulting  evolu¬ 
tionary  equation  using  their  constitutive  relationships. 
The  non-lossy  part  gives  the  dielectric  function  relat¬ 
ing  D  to  E,  and  the  lossy  parts  give  the  magnetisation 
and  conduction  currents,  jp.- 

D  =  €E,  ji  =  -V  X  /35  -1-  (r£  (4) 

where  (3  is  the  inductive  loss  coefficient  and  cr  is  the 
conductivity. 

A  small  magnetisation  current  can  be  used  in  sim¬ 
ulations  to  control  numerical  noise,  thereby  allowing 
simulations  to  be  performed  with  fewer  simulation  su¬ 
perparticles  than  would  otherwise  be  possible. 

2.3.  Electromagnetic  boundary  conditions 

2.3.1.  Internal 

Internal  boundary  conditions  are  used  to  treat  pe¬ 
riodicity,  symmetry  and  to  allow  domain  decompo¬ 
sition.  The  latter  case  arises  when  the  computational 
domain  is  subdivided  into  blocks  (cf.  Section  3.2). 
Internal  boundary  conditions  are  implemented  using 
“glue-patches”  (Section  4.1.6). 

2.3.2.  External 

External  boundary  conditions  specify  tangential 
electric  fields  or  normal  magnetic  fields.  They  are 
typically  Dirichlet  conditions,  and  may  apply  at  sur¬ 
faces  completely  embedded  within  the  computational 
domain  (e.g.  an  internal  conductor). 

The  full  wave  description  within  a  device  is  linked 
to  a  lumped  circuit  approximation  of  power  supplies 
and  extraction  waveguides  by  coupling  elements  at  the 


surface  of  the  active  computational  domain  to  external 
circuit  elements.  The  external  elements  have  no  physi¬ 
cal  dimension,  but  they  provide  a  relationship  between 
surface  current,  /,  and  tangential  electric  fields,  E. 
Thus,  a  perfect  conductor  would  give  E  =  0,  a  purely 
resistive  circuit  element  would  give  E  =  Zl  and  a  cir¬ 
cuit  with  resistance  and  inductance  (or  capacitance) 
would  give 

=  +  (5) 

dt  dt 

where  circuit  coefficients  ac.  7c>  are  chosen 
to  describe  the  particular  external  circuit  (see  [  18] ). 
Similarly,  a  second  order  differential  equation  is  used 
for  coupling  to  an  L-R-C  circuit  and  so  forth.  For 
steady  state  circuits,  Eq.  (5)  can  be  replaced  by  the 
complex  impedance  equation  E  =  Zl. 

Radiation  boundary  conditions  can  be  treated  using 
the  volume  wave  absorption  method  used  in  MAGIC 
[8,9] .  This  involves  the  addition  of  a  non-physical  ab¬ 
sorbing  medium  (giving  terms  -aE  in  Ampere’s  Law 
and  -anB  in  Faraday’s  Law)  at  free  space  bound¬ 
aries. 

2.4.  Particle  boundary  conditions 

2.4.1.  Internal 

Internal  particle  boundary  conditions  follow  the 
same  classification  as  field  boundary  conditions,  and 
like  their  field  counterparts  are  implemented  using 
glue-patches.  These  boundary  conditions  conserve 
particle  number  and  involve  transformation  of  particle 
position  and/or  momentum  coordinates. 

2.4.2.  External 

External  particle  boundary  conditions  specify  ei¬ 
ther  particle  absorption  or  emission;  for  example  beam 
injection,  cathode  emission  and  secondary  emission. 
Electron  emission  from  boundary  surfaces  is  treated 
using  standard  Monte-Carlo  procedures.  Beam  injec¬ 
tion  selects  superparticle  positions  and  momenta  to  fit 
the  incoming  beam  distribution  function.  Space  charge 
limited  emission  at  cathodes  is  modelled  by  introduc¬ 
ing  sufficient  free  surface  space  charge  (in  the  form 
of  superparticles)  to  bring  the  normal  electric  field  at 
the  surface  to  zero.  Secondary  electrons  at  boundary 
surfaces  are  generated  and  emitted  in  response  to  the 
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locations  and  momenta  of  incident  superparticles  to  fit 
the  chosen  energy  and  direction  distribution. 


3.  Numerical  scheme 


The  permutation  symbols  e'^’^  and  Cijk  are  1  for 
cyclic  indices,  -1  for  anticyclic  indices,  and  0  other¬ 
wise,  and  are  related  to  the  permutation  tensors  by 

=  ^'>7  75;  eijk  =  ^eijk  ( 9 ) 


3.1.  Tensor  formulation 


This  section  summarises  the  definitions  and  iden¬ 
tities  in  general  curvilinear  coordinates  [19]  used  in 
the  algorithm  derivation  and  formulation. 

Let  the  reference  Cartesian  coordinates  have  com¬ 
ponents  {.x\x-,x'^)  and  unit  vectors  so 

that  a  vector,  x,  may  be  written  x  =  x‘xj,  where  sum¬ 
mation  over  the  repeated  index  i  (=  1,  2,  3)  is  im¬ 
plied.  If  (.v',  are  expressed  as  functions  of  the 

curvilinear  coordinate  components  (x\x^,x^),  then 
x'  =  .v‘(.r',A-,x^),  etc,  or  more  concisely  a  =  a(a). 

The  basis  vectors  and  the  reciprocal  basis  vec¬ 
tors  e'  are  defined  by 

dx  ■  ■ 

These  vectors  are  orthogonal;  e,  ■  eJ  =  S{  where  5‘j 
is  the  Kronecker  delta  (=  1  if  /  =  j,  0  otherwise). 
The  reciprocal  basis  vectors  can  be  written  in  terms 
of  the  basis  vectors  e'  =  ej  x  e^j  J  and  vice-versa  c,  = 
{e’  X  e^)J  where  the  Jacobian  /  =  75  =  |e,- .  x  ek\ 
is  the  square  root  of  the  determinant  g  of  the  metric 
tensor. 

The  (covariant)  metric  tensor  is  defined  as  = 
Bi-Bj  and  its  reciprocal  tensor,  the  contravariant  metric 
tensor  is  7^  =  e'  •  ef  In  Cartesian  coordinates,  the 
covariant,  contravariant  and  physical  components  are 
identical  and  the  metric  tensor  reduces  to  g/y  =  5,y .  For 
orthogonal  systems,  gij  =  0  for  i  ¥=  j.  In  general  gjj 
is  symmetric  with  six  distinct  elements,  =  gy,-. 


3.7.7.  Vector  identities 

A  vector  A  may  be  expressed  in  terms  of  its  con¬ 
travariant  component  A‘,  covar/a«f  components  A,  or 
physical  components,  A{i) 


A  =  A'e,  =  A,e'  =  A(i)ei 

where 

ei 

^-■  =  777 


(7) 

(8) 


The  permutation  tensors  satisfy  the  identity 
^  ^  ^kim  =  S‘i&i„  —  (10) 

Vector  dot  and  cross  products  become: 


O’  b  =  a'bi  =  aib' 
{a  X  by  =  e'j^ajbk 


(ax  b)i  =  €ijka^b'‘ 

(11) 

The  common  vector  operations  are: 

(Vp),  =  |5 

dx' 

(12) 

r-,  1^^/ 

V-  a  =  — — 
s/g  dx'  '' 

(13) 

(V  X  a)'  = 

dxJ 

(14) 

^  i  d 

a‘Vp  =  a‘—p 
dx' 

(15) 

(16) 

3.1.2.  Maxwell’s  equations 
Using  the  formulae  of  Section  3.1.1  we  can  write 


Maxwell’s  equations  in  tensor  form:- 

dt  dxl 

(17) 

(18) 

dD‘ _.j,dHk  ., 
dt  dxl  •' 

(19) 

(20) 

It  is  convenient  to  introduce  extensive  current  and 
charge  variables 


I‘=z./Zi‘-  n 


J.W.  Eastwood  et  al. /Computer  Physics  Communications  87  (1995)  155-178 


159 


and  volume  scaled  flux  quantities  b'  and  d'  for  mag¬ 
netic  and  displacement  fields 

b'  =  y/gB‘-  d‘  =  ^D‘  (22) 

If  in  addition  we  write  the  permutation  tensor  in  terms 
of  the  permutation  symbol.  Maxwell’s  equations  be- 


come 

dt  dxJ 

(23) 

Sb  1  Cb 

bi  ^ 

II 

o 

(24) 

^  _  f 

dt  dxi 

(25) 

dx‘ 

(26) 

Eqs.  (23)-(26)  have  the  particularly  attractive  fea¬ 
ture  that  they  have  the  same  form  irrespective  of  coor¬ 
dinate  system,  and  contain  no  explicit  reference  to  the 
metrics.  This  is  also  true  of  the  relationships  between 
the  electromagnetic  fields  and  potentials: 


dd) 

(27) 

dxi 

(28) 

The  only  explicit  reference  to  the  metrics  appears  in 
the  constitutive  relationships  between  fields,  which  in 
vacuo  are 


pioH.  =  ^bi;  eoEi  =  ^d^  (29) 

y/g  y/g 

More  generally,  /Uo  and  eo  are  replaced  by  tensor  per¬ 
meabilities  and  permittivities. 

A  similar  attractive  simplicity  appears  in  the  Action 
Integral  for  the  electromagnetic  field  equations: 

1  =  dt  dx'  dx^  dx^ 

X  Q (£,^‘  -  Hib‘)  +  I‘Ai  -  (2<^^  (30) 

This  metric-free  form  simplifies  the  problem  of  writing 
a  program  module  for  solving  Maxwell’s  equations  in 
arbitrary  non-orthogonal  coordinate  systems. 


3.1.3.  Equations  of  motion 

The  relativistic  equations  of  motion  of  a  charged 
particle  are 


(31) 


^  =  qiE  +  vxB)  (32) 

dt 

where 


p  =  ymov 
7^  =  1  -H 


=  1/1- 


(33) 

(34) 


In  general  coordinates  (^',^^,x^),  these  become 
dx'‘ 


dt 


(35) 


and 

^-r:iPrv'  =  q[E,  +  e,irv'b^]  (36) 

at 

where  the  Christoffel  symbol  of  the  first  kind  is  given 
by 


r'jh 


(37) 


The  particle  dynamics  can  be  included  in  the  ac¬ 
tion  integral,  Eq.  (30)  by  adding  the  extra  particle 
Lagrangian  term 


/ 


'^dt 

y 


(38) 


and  explicitly  expressing  the  charges  and  currents  in 
Eq.  (30)  as  sums  over  particles,  i.  with  charges  qi 


Q  =  ^qMx!  -  x')S(xf  -  x^-)S(x^  -  x^)  (39) 

/ 

r  =  Y^q,8(x]  -x^)S{x]-x-)8{x]-x^)-jf 

(40) 


The  equation  of  motion  Eq.  (36)  arises  from  the 
Euler-Lagrange  equation 


dx^ 


dtdi' 


where  L  is  the  total  Lagrangian  density. 


(41) 
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3.2.  Element  generation 

3.2.1.  Multiblock  decomposition 

The  elements  are  generated  using  a  multiblock  de¬ 
composition;  complex  objects  are  divided  into  a  set 
of  curvilinear  hexahedral  blocks  (where  each  face  has 
four  edges),  which  in  turn  are  subdivided  into  hexa¬ 
hedral  finite  elements  using  transfinite  interpolation. 
Metric  tensors  and  basis  vectors  are  defined  on  each 
element  by  using  coordinate  transformations  isopara¬ 
metric  to  the  electric  potential  representation. 

The  user  of  the  software  controls  the  multiblock  de¬ 
composition  by  describing  objects  in  terms  of  a  “kit- 
of-parts”  (Section  4.2).  In  selecting  the  parts  for  the 
multiblock  division,  complex  objects  are  divided  into 
sufficiently  convex  volumes  to  avoid  the  creation  of 
very  small,  or  even  negative,  volume  elements  in  the 
blending  process;  this  could  cause  the  numerical  sim¬ 
ulations  to  fail.  However  in  the  applications  envisaged 
it  is  unlikely  that  singular  elements  will  present  seri¬ 
ous  difficulties.  Computational  efficiency  dictates  that 
wherever  possible,  orthogonal  blocks  (such  as  rect¬ 
angular  bricks  and  cylinder  sections)  be  used  as  this 
greatly  reduces  computational  costs  and  storage.  To 
facilitate  load  balancing  on  MIMD  computers,  it  may 
also  be  advantageous  to  subdivide  blocks  beyond  the 
level  dictated  by  the  object’s  geometry. 

Each  block  of  the  multiblock  decomposition  is 
described  by  its  bounding  curves;  four  intersecting 
curves  in  two  dimensions  and  twelve  curves  in  three 
dimensions.  Alternatively,  the  3-D  block  may  be  de¬ 
scribed  by  its  bounding  surfaces.  The  relationship  of 
the  block  coordinates  to  the  global  coordinates  of 
the  object  is  described  in  Section  3.5.  The  locations 
of  element  nodes  within  a  block  are  generated  by 
blending  the  interpolants  from  the  bounding  curves 
or  surfaces,  as  described  in  Sections  3.2.3. 

In  order  to  transform  between  curved  space  (barred 
variables)  and  physical  coordinates,  and  to  integrate 
the  equations  of  motion,  values  of  the  basis  vectors 
are  required.  The  integration  of  the  field  equations 
and  transformation  between  covariant  and  contravari- 
ant  components  requires  metric  tensor  values.  These 
may  be  evaluated  from  the  finite  element  approxima¬ 
tions  to  the  coordinate  transformations,  as  outlined  be¬ 
low. 


3.2.2.  Isoparametric  elements 
An  isoparametric  element  uses  the  same  function 
for  the  coordinate  mapping  as  for  the  finite  element 
support  of,  in  this  case,  the  scalar  potential  0.  Fig.  1 
shows  an  isoparametric  quadrilateral  element  in  phys¬ 
ical  and  barred  coordinate  space.  A  point  x  is  related 
to  the  barred  coordinates  x  =  (x', x^)  by 

a:  =  xi(1  -  x')(l  -  .r)  +  X2x\  \  -r) 

+x-ix^x- +  Xi{\  -  (42) 

This  gives  a  basis  vector 

dx  ,  ,  T 

e\  =  =  {X2  -  xx){  \  -  x^)  +  (xj  -  X4)x- 

=  rs(l  -  X-)  +  rnx'^  (43) 

and  similarly  for  e2,  from  which  the  reciprocal  basis 
vectors,  metric  tensor  elements  and  Jacobian  can  be 
readily  computed  at  any  point  within  the  element. 

Basis  vectors  and  metrics  are  defined  in  the  same 
manner  for  3-D  hexahedral  elements.  If  we  label  the 
nodes  of  the  element  by  the  index  triplet  ( /,  j,  k) , 


i,j.  A:  =  0  or  1,  and  let 

wo(z)=(l-z),  wi(z)  =  z  (44) 

then  the  analogue  of  Eq.  (42)  is 
X  =  XijkWi{x')wj(f-)wi,(x?)  (45) 

and  the  basis  vectors  are 

=  (Xljk  -  X0jk)Wjix-)Wkix^) 

=  f^jkWj(x^)wk(x-)  (46) 

^2  =  r,iiVV,(x')wt(x^)  (47) 

e3  =  rjjiWi(x')wj(f-)  (48) 


The  reciprocal  basis  vectors  and  metric  tensor  ele¬ 
ments  follow  by  substituting  £,•  fromEqs.  (45)-(48) 
into  the  appropriate  formulae  from  Section  3.1. 

3.2.3.  Transfinite  interpolation 
Mesh  generation  for  finite  volume  and  finite  element 
schemes,  by  blending  the  interpolants  from  intersect¬ 
ing  pairs  of  curves  such  that  the  mesh  exactly  fits  the 
boundary  curves,  was  introduced  by  Gordon  and  Hall 
[20]  and  is  now  widely  used  in  body-fitted  fluid  flow 
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Fig.  1.  The  isoparametric  linear  quadrilateral  element  with  comer  nodes  x, . .  .xr  maps  to  the  unit  square  in  the  barred  coordinate  space. 


ally,  the  functions  A,  ^  and  r]  are  chosen  to  be  linear. 


Oil 

001 


s  000 


100 


Fig.  2.  The  six  faced  volume  is  defined  by  the  twelve  bounding 
curves  joining  the  comer  nodes  at  positions  x,;*,  i.y,  k  =  0  ot  1 

codes,  such  as  Harwell-FLOW3D  [21,22].  We  sum¬ 
marise  here  the  transfinite  interpolation  formulae  that 
we  use  in  our  body-fitting  software. 

The  aim  is  to  divide  the  general  curvilinear  hexa- 
hedra  as  illustrated  in  Fig.  2  into  a  set  of  hexahedral 
elements  which  map  to  unit  cubes  in  curved  space. 
The  bounding  curve  Xijir),  re[0, 1]  joins  the  cor¬ 
ner  node  xoij  at  r  =  0  to  node  xuj  at  r  =  1.  Similarly 
Yijis)  joins  XiOj  to  xnj  and  Zy(f)  joins  Xyo  to  Xyi, 
where  /, ;  =  0  or  1 .  The  interpolation  functions  for  the 
r.  5,  and  t  coordinates  are  respectively  A,  and  77. 

Blending  interpolants  between  the  bounding  curves 
to  obtain  exact  fits  at  all  eight  edges  gives  the  formula 

xir,s,t)=Xjkir)i//j(s)T]k(l) 

+Yik{s)\i{r)Tjk{t) 

-2xijk\i{r)ipj{s)7]k{t)  (49) 

where  sums  over  i,],  k  =  0  and  1  are  implied.  Gener- 


3.3.  Maxwell’s  equations 

The  discrete  finite  element  approximations  to 
Maxwell’s  equations  are  obtained,  following  Refs. 
[4,15] ,  using  virtual  particle  electromagnetic  particle- 
mesh  schemes  for  general  3-D  coordinate  systems. 
The  present  implementation  uses  the  lowest  order 
conforming  elements,  and  employs  lumping  to  main¬ 
tain  explicit  time  integration.  If  more  accurate  wave 
dispersion  is  required,  then  the  lumping  approxima¬ 
tions  could  be  replaced  by  “mass”  matrix  inversions. 

Discrete  equations  are  derived  by  introducing  finite 
element  approximations  to  the  potentials 

(f)=:4>U  (50) 

=  (51) 

where  <P  and  A(,)  are  nodal  amplitudes.  Eq.  (50)  is 
shorthand  (Eq.  (51)  similarly)  for 

<f,ix\  x\  t) 

=  Y^^ik,  n)Uik,  n;  x\  .X-,  x\  t)  (52) 

where  the  sum  is  over  spatial  (k)  and  temporal  («) 
node  indices.  The  approximations  (50)  and  (51)  are 
substituted  into  the  Action  Integral  Eq.  (30),  which 
is  then  varied  with  respect  to  the  nodal  amplitudes  <P 
and  A  to  yield  discrete  approximations  to  the  inhomo¬ 
geneous  Maxwell’s  Eqs.  (25)  and  (26).  (The  homo¬ 
geneous  Maxwell  Equations  (23)  and  (24)  follow  by 
virtue  of  Eqs,  (27)  and  (28),  if  V  and  W,-  are  appro¬ 
priately  chosen.) 
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3.3.1.  Finite  element  field  approximations 
The  lowest  order  conforming  choice  of  element  sup¬ 
port  for  the  potentials  is  a  mixture  of  piecewise  lin¬ 
ear  and  piecewise  constant  functions.  Eqs.  (27)  and 
(28)  are  satisfied  in  a  strong  sense  for  the  lowest  order 
conforming  elements  provided  that  the  finite  element 


approximations  to  the  fields  are 

=  (53) 

=  (54) 

b'  =  (55) 

H  =  ^(i)^(i)  (56) 

where 

Wi{x\x-,x^,t)  =  n{x‘)Aix-’)Aix‘^)A(t)  (57) 

U{x\x^,x\t)  =  A{x')A{x-)Aix^)n(t)  (58) 

Vi{x\x^,x\t)  =  n{x‘)A{x^)A{x'')n{t)  (59) 

Xi{x\x-,x^  ,t)  =  A{x‘)n{x^)n{x’')A{t)  (60) 


Fig.  3.  The  location  of  nodes  on  the  unit  cube  element.  Crosses 
give  A,,  E;,  I'  and  d'  node  locations,  open  circles  give  H,-  and 
b'  locations  and  solid  circles  give  element  comer  and  0  node 
locations. 


gives 


dt  dxi 


(64) 


Using  the  above  definitions  of  X,-  and  V,  shows  that  Eq 
(64)  reduces  to  the  standard  Yee  [13]  finite  difference 
form  of  Faraday’s  law,  but  for  nodal  amplitudes  of 
field  components  b'  and  E*-. 


[  0  ;  otherwise 

nix')  =  I  i 

[  0  ;  otherwise 

and  (i,  j,  k)  are  cyclic  permutations  of  indices  (1,2, 
3).  Fig.  3  shows  the  location  of  the  nodes  at  which 
amplitudes  of  these  finite  element  field  approxima¬ 
tions  are  defined.  The  following  two  subsections  dis¬ 
cuss  how  this  choice  of  FE  approximation  yields  field 
equations  of  a  similar  form  to  the  familiar  staggered 
leapfrog  finite  difference  equations  [13]  and  charge 
assignment  similar  to  the  standard  CIC  scheme  [1,2]. 
Higher  order  schemes  could  be  generated  by  using 
higher  order  FE  support  functions,  but  are  not  consid¬ 
ered  further  here. 


(61) 

(62) 


3.3.2.  Homogeneous  equations 
The  homogeneous  equations  follow  immediately 
from  the  definitions  of  the  finite  element  fields  and 
Eqs.  (23),  (24),  (27)  and  (28).  Magnetic  flux  is 
identically  conserved  and  Faraday’s  law 


dt  dxl 


(63) 


3.3.3.  Inhomogeneous  equations 

Ampere’s  equation  is  obtained  by  assembling  con¬ 
tributions  to  the  variation  of  the  Action  from  adjacent 
elements 

^^=Jdtdx^  df-dx^!^-d‘^V,o 

_H  }  =  0  (65) 

The  current  term,  which  gives  the  prescription  for  as¬ 
signing  current  from  particles  to  the  elements,  is  dis¬ 
cussed  further  in  Section  3.4.1. 

For  each  node  internal  to  a  block,  there  are  eight 
contributions  (from  four  adjacent  elements  at  two 
timelevels)  to  the  assembled  equation  resulting  from 
Eq.  (65)  for  a  displacement  field  component.  At 
block  surfaces,  this  number  may  be  different.  In  the 
implementation  of  the  software,  element  contribu¬ 
tions  are  pre-assembled  for  nodes  internal  to  blocks, 
but  contributions  across  block  faces  are  assembled  by 
“glue-patch”  data  exchanges  (cf.  Section  4.1.6)  only 
during  the  computation  of  the  displacement  fields. 
For  surface  nodes,  there  are  contributions  from  ele¬ 
ments  within  the  computational  volume,  plus  possibly 
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surface  current  terms  from  the  external  boundary 
conditions. 

Variation  of  I  with  respect  to  the  potential  nodal  am¬ 
plitudes  gives  the  discrete  approximations  to  Gauss’ 
Law: 

§  ■=  /* 

=  0  (66) 

The  last  term  gives  the  prescription  for  charge  assign¬ 
ment,  and  the  remaining  part  of  the  integral  gives  con¬ 
tributions  to  the  divergence  of  the  displacement  field. 
As  for  Ampere’s  equations,  the  number  of  contribu¬ 
tions  to  a  node  at  a  block  surface  may  differ  from  that 
for  nodes  interior  to  blocks,  and  boundary  conditions 
are  applied  via  surface  patches. 

For  simplicity,  the  present  implementation  of 
the  software  uses  the  lumped  approximations 
('^(0-  ^(o)  =  1’  (^(0’  ^(o)  =  1-  where  the  inner 

product  notation 

{a,b)=  J dt  dx^  dx-  d.x^a(x‘,t)b(x‘,t)  (67) 

D 

is  used  for  the  integral  over  the  domain  of  interest  D. 
The  advantage  of  lumping  is  that  it  greatly  reduces  the 
amount  of  computational  work  per  node  per  timestep, 
but  at  the  cost  of  degrading  dispersive  properties  and 
introducing  the  possibility  of  numerical  Cerenkov  in¬ 
stability  [23].  With  lumping,  the  assembled  equation 
resulting  from  Eq.  (65)  becomes 

S,d‘ =  e‘j'^djy\k-\‘  (68) 

where  the  operators  d  denote  centred  difference. 
Lumping  in  time  has  the  advantage  of  making 
timestepping  explicit. 

3.3.4.  Constitutive  relations 

For  isotropic  permeability  and  permittivity  tensors 
the  finite  element  approximations  to  the  constitutive 
relations  are 

(E,V„).  V(0)  =  (69) 

and 

X(o)  =  (^g/yiyXo),  X(o)  (70) 


Fig.  4.  Values  of  C\2  at  the  NW,  NE,  SW  and  SE  nodes  couple 
values  of  d',  Ei,  and  the  N  and  S  nodes  to  d’,  Ei  at  the  E  and 
W  nodes. 

Using  lumped  approximations  these  reduce  to 

E,  =  G^d^'  (71) 

H,-  =  G,^iy  (72) 

Elements  of  the  symmetric  tensors  G^  and  G-J  are 
sparse  matrices.  More  general  permittivities  and  per¬ 
meabilities  are  handled  by  replacing  e  and  /x  by  tensor 
functions  in  the  computation  of  G,^  and  G^,  respec¬ 
tively. 

The  constitutive  relations  (71)  and  (72)  resulting 
after  assembly  at  an  internal  node  generally  have  four 
off-diagonal  terms  for  each  element.  Fig.  4  illustrates 
the  location  of  the  nodes.  For  example,  contributions 
from  the  element  to  Ei  at  the  W  node  are 

5Ei=Gfj(W)d'(W)  +Gf2(AW)d-(V) 

-bGf2(5W)d^5)  (73) 

and  to  H2  at  the  W  node 

5H2  =  G^2(  6-(  +  Gf2  ( NW)  b'  ( N) 

+Gf2(6W)b'(5)  (74) 

Similar  expressions  hold  for  other  components  at  the 
other  nodes.  Adding  contributions  of  all  elements  to 
a  given  node  implies  a  four  point  sum  for  the  off- 
diagonal  terms. 

3.3.5.  Metric  computations 

The  metric  terms  are  defined  by  (no  implied  sum¬ 
mation) 


(75) 
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and 

=  (76, 

The  integrals  in  Eqs.  (75)  and  (76)  are  evalu¬ 
ated  element-by-element  in  the  preprocessor  ( Section 
4.2.1)  using  Gaussian  quadrature.  This  metric  com¬ 
putation  assembles  the  terms  within  the  interior  of  a 
uniblock,  but  leaves  the  surface  coefficients  unassem¬ 
bled  to  facilitate  patch  exchange. 


3.3.6.  Electromagnetic  boundary  conditions 
The  internal  electromagnetic  boundary  conditions 
outlined  in  Section  2.3  are  dealt  with  by  glue-patch 
exchange  (Section  4.1.6),  and  the  external  boundary 
conditions  described  in  this  section  appear  as  extra 
terms  in  the  evolutionary  equations  for  fields  at  surface 
nodes. 

For  a  surface  of  a  block  arranged  so  that  its  normal 
is  parallel  to  e^,  the  vector  equivalent  of  Ohm’s  Law 
ZI  =  E  is 

X  (ZIs  -  E)  =0  (77) 

where  the  surface  current  has  l]  =  0.  Now  for  an 
arbitrary  vector  /,  straightforward  manipulation  gives 

e^xf=V8{e'(-g^^f  +  g^^f) 

+eHg^^f  -  g' V) 

+e\-g^^f+g'^f)}  (78) 

Taking 
f  =  ZIs-E 


and  noting  that 


f  =  -E^  = 


-d^ 


(79) 


it  follows,  from  equating  the  components  of  Eq.  (78) 
separately  to  zero  that 


r  = 


i=  1,2 


(80) 


Geometrical  considerations  lead  to  the  result  that  the 
volume  current  is  given  by 


I  =  Is\  ei  X  €2 


(81) 


Algorithmically,  the  first  term  on  the  right-hand  side 
of  Eq.  (80)  presents  few  problems.  The  second  van¬ 
ishes  if  the  3-coordinate  is  normal  to  the  surface  in 
the  sense  that  g,3  =  g2y  =  0,  since  g'jgjy  =  0  for  f  =  1, 
2.  Alternatively,  if  the  coordinates  within  the  surface 
are  orthogonal  so  that  gi2  =  0,  then  the  relations 


6.33 


g,3 

g(i)U)  ’ 


/=  1,2 


(82) 


apply  and  g‘^d^/g^^  can  be  replaced  by 
G?d^ 


Si  =  - 


^(0(0 


(83) 


The  d'  on  a  resistive  wall  patch  are  thus  updated  using 
d'^"-*-')  =  -1-  ygd''"’  -b  Yi,  i=  1,2  (84) 


where  n  and  n  -f  1  denote  time  levels.  Representing  d‘ 
in  Eq.  (80)  as  -t-  ( 1  —  ^)d'^''^  it  follows  that 


(X  =  - = - 

Z  +  20 

(85) 

13= 

z  +  2e 

(86) 

and 

2Si 

Z  +  20 

(87) 

where 

2= 

(88) 

Note  that  provided  0  >  1/2  the  above  can  be  used  to 
represent  a  perfectly  conducting  wall.  Boundary  con¬ 
ditions  involving  the  specification  of  normal  magnetic 
field  and  tangential  electric  field  are  straightforwardly 
implemented;  the  latter  are  in  many  cases  equivalent 
to  specifying  a  potential  difference  across  a  block. 


3.3.7.  Timestep  loop 

The  timestep  loop  for  the  integration  of  the  field 
equations  deals  exclusively  with  field  components.  In¬ 
put  from  the  particle  integration  are  the  contravariant 
currents  l‘,  and  the  output  to  the  particle  integration 
are  field  components  E,-  and  b'.  Details  of  the  particle 
integration  and  boundary  condition  will  be  discussed 
later.  The  finite  element  field  equations  are  assembled 
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within  each  block,  and  glue-patch  data  exchange  is 
used  to  complete  the  assembly  at  block  surfaces.  The 
electromagnetic  field  integration  routines  advance  the 
primary  field  arrays  -  the  set  of  nodal  amplitudes  of 
d'  and  b'  -  by  the  electromagnetic  integration  timestep 
(<  particle  timestep).  This  leads  to  the  following  op¬ 
erations  in  the  timestep  loop,  starting  from  currents  I' 
and  fields  b'  at  time  level  n+  1/2  and  displacement 
fields  at  time  level  n: 

(i)  obtain  H,  from  b'  in  each  block  (Eq.  (72)). 

(ii)  compute  the  contributions  d'  to  the  displace¬ 
ment  current  in  each  block,  using  the  right  hand 
side  of  Eq.  (68)  for  internal  nodes,  and  its  par¬ 
tially  assembled  equivalent  for  surface  nodes. 

(iii)  assemble  d'  contributions  at  block  surfaces  by 
glue-patch  data  exchange.  This  step  applies  the 
‘•internal”  boundary  conditions  (of  periodicity, 
symmetry,  neumann  type  and  domain  decom¬ 
position). 

(iv)  update  d'  and  apply  external  boundary  condi¬ 
tions.  At  internal  boundaries,  the  new  d'  is  com¬ 
puted  from  d,d‘  =  d'  .  The  treatment  of  external 
nodes  is  described  in  Section  3.3.6. 

(v)  obtain  E,-  from  d'  in  each  block  (Eq.  (71)). 

(vi)  assemble  E,  contributions  across  internal 
boundaries  by  glue-patch  data  exchange. 

(vii)  apply  external  boundary  conditions  to  E, 

(viii)  advance  b'  in  each  block  (Eq.  (64)). 

At  the  end  of  the  electromagnetic  timestep,  nodal 
values  d'  are  known  at  time  level  n  -t-  1  and  b'  are 
known  at  time  level  n  +  3/2. 

3.4.  Particle  equations 

The  electromagnetic  field  equations  take  a  particu¬ 
larly  simple  form  when  the  appropriate  choices  of  co- 
and  contra-  variant  fields  are  made;  the  same  is  true 
for  current  assignment,  force  interpolation  and  the  in¬ 
tegration  of  the  particle  equations  of  motion.  The  only 
substantial  difference  between  the  algorithm  in  gen¬ 
eral  geometry  and  in  Cartesian  coordinates  arises  in 
the  update  of  particle  positions. 

3.4.1.  Assignment 

The  approximation  of  representing  the  distribution 
function  by  a  set  of  “superparticle”  sample  points  re¬ 
duces  the  velocity  space  integrals  for  charge  and  cur¬ 
rent  density  to  sums  over  particles  (Eqs.  (39)  and 


(40)).  Substituting  these  sums  into  the  source  term 
integrals  arising  from  the  variations  of  the  field  La- 
grangian  with  respect  to  (P  and  A,  give  respectively 
the  prescriptions  for  charge  assignment 

j  dtY^qpU{x\,xl,xl,t)  (89) 

and  for  current  assignment. 


The  integrals  for  Q  and  I'  are  evaluated  in  exactly  the 
same  manner  as  described  in  Ref.  [15].  Motion  is 
assumed  to  be  impulsive  and  trajectory  segments  ^e 
assumed  to  be  straight  lines  in  the  curved  (x  ,x  ,  x  ) 
space  in  which  elements  are  unit  cubes.  The  sums  in 
Eqs.  (89)  and  (90)  are  over  particles  p.  Since  the 
expressions  are  linear,  there  is  no  loss  of  generality  in 
considering  a  single  particle  of  charge  q. 

Charge  assignment.  If  we  label  nodes  on  the  unit  cube 
element  in  curved  space  ( i,  j,k)  as  shown  in  Fig.  5  and 
write  the  assignment  function  in  terms  of  its  product 
parts 

Uijk{x\x'^,x^,t)  =Wi(,x')wjix-)wk{x^)  (91) 

the  charge  assigned  to  element  node  {i,j,k)  where 
i,j,  it  =  0  or  1  becomes 

Qijk  =  qwiix^)wj{x^)wk{x^)  (92) 

where  u'o(x)  =  \  —  x,  w\  (x)  =  x  and  (x*  ,x-,x  )  is 
the  displacement  of  the  particle  position  from  the  ele¬ 
ment  corner  (0,0,0).  Inspection  of  Eq.  (92)  reveals 
the  assignment  scheme  to  be  the  same  as  area  weight¬ 
ing  [  1,2] ,  but  now  applied  on  a  uniform  net  in  curved 
space  rather  than  in  Cartesian,  physical  space.  Locat¬ 
ing  the  element  containing  the  particle  is  trivial,  since 
the  element  index  is  given  by  the  integer  truncation 
of  the  particle  coordinates  x‘,  exactly  as  for  simple 
Cartesian  PIC. 

Current  assignment.  Using  the  impulse  approximation 
to  particle  motion  as  described  in  [15]  and  the  product 
form  of  W  reduces  Eq.  (90)  for  the  contribution  of  a 
single  particle  to 
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Fig.  5.  (a)  Charge  assignment  from  a  single  particle  and  (b)  I'  current  assignment  from  a  single  panicle  trajectory  segment  for  linear 
potential  test  functions  in  physical  space. 


1/2 

dsqsAx^WjiX^  +  s^x-)w^,{X^  + 

-1/2 

(93) 

The  quadratic  integral  is  evaluated  exactly  by  point 
Gaussian  quadrature,  so  Eq.  (93)  can  be  written  as 
the  contribution  from  “virtual  particles”  at  a:±:- 

'U-  +  Wj{xl)Wk{xl)] 

(94) 


and  similarly  for  1^  and  1^  by  cyclic  permutation  of 
indices. 

3.4.2.  Charge  conservation 
By  linear  superposition,  conservation  for  one  tra¬ 
jectory  moving  through  a  single  time  step  implies  the 
same  for  the  sum  of  all  trajectories.  Summing  all  con¬ 
tributions  to  Eqs.  (93)  from  a  single  particle,  gives  the 
same  as  the  difference  of  Eqs.  (92)  at  start  and  end 
points,  i.e.  the  linear  hexahedral  element  case  satisfies 

d,Q  =  -41^  (97) 


where  indices  j  and  k  take  values  0  or  1.  The  positions 
of  the  virtual  particle  have  coordinates 

where  X'‘  =  (x*  -f  )/2,  ^x‘‘  =  -  xf,  and  k  = 

1,  2  or  3.  Coordinates  (xf,xf)  and  (^^,5})  are  re¬ 
spectively  the  start  (i)  and  end  (/)  of  the  trajectory 
segment.  The  corresponding  expressions  for  current 
components  and  are  given  by  simultaneously 
cyclically  permuting  component  indices  ( 123)  and 
node  indices  (^k)  in  Eq.  (94).  Fig.  5  gives  a  Carte¬ 
sian  space  sketch  of  assignment  according  according 
to  (a)  Eq.  (92)  for  charge  and  (b)  Eq.  (94)  for  cur¬ 
rent. 

In  practice,  it  more  convenient  to  evaluate  explicitly 
the  integrals  for  I'  rather  than  use  quadrature.  Eq.  (94) 
then  becomes 


where  the  symbol  <9  denotes  a  centred  difference  aris¬ 
ing  from  assembling  the  finite  element  contributions, 
and  Q  and  I*  are  nodal  amplitudes.  Equations  of  the 
form  (97)  can  be  shown  to  be  generally  satisfied  for 
virtual  particle  algorithms. 

The  great  benefit  of  Eq.  (97)  is  computational 
speed-up;  charge  assignment  and  solution  of  Poisson’s 
equation  for  the  electrostatic  potential  correction  are 
not  needed  in  the  timestep  loop. 

3.4.3.  Force  interpolation 

Fields  at  particle  positions  are  given  by  the  finite 
element  fields  (Eqs.  (53)-(56))  sampled  at  the  par¬ 
ticle  positions,  where  the  interpolation  functions  are 
given  by  Eqs.  (57)-(62). 

3.4.4.  Momentum  equation 

Particle  momenta  and  positions  are  stored  in  terms 
of  their  curvilinear  coordinates  local  to  the  block 
which  contains  them.  The  momentum  coordinates  are 
updated  in  two  stages.  First,  the  momentum  compo¬ 
nents  are  updated  at  the  old  particle  position,  then 
are  transformed  to  components  measured  at  the  new 
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position;  this  eliminates  the  explicit  appearance  of  the 
Christoffel  symbols  terms  in  the  discrete  approxima¬ 
tion  to  Eq  (36). 

The  established  centred  time  approximation  [7,1,2] 
is  used  for  the  momentum  update  at  the  old  particle 
position.  Covariant  momentum  components  k  at  time 
level  n-b  1  measured  at  the  current  particle 

positions  are  found  using 

(«+l/2)  -(n-i/2) 

Pk  ~  Pk 

(98) 

where  v'  =  lyniQ  are  contravariant  velocity  compo¬ 
nents  and  eum  is  the  permutation  symbol.  Eq.  (98) 
is  solved  using  the  method  due  to  Boris  [7];  apply  a 
half  electric  acceleration,  compute  the  relativistic  y, 
apply  a  two  stage  rotate  and  then  complete  the  elec¬ 
tric  acceleration.  Fields  are  evaluated  at  the  particle 
positions  using  Eqs.  (54)  and  (55). 

3.4.5.  Position  equation 

Positions  are  updated  to  new  time  level  n  -t-  1  and 
the  momentum  components  at  the  new  positions  are 
computed  using  a  predictor-corrector  scheme  to  solve 

^kin+\)  _  ^k(n) 

=  ^i;*("+i/2)(^(«))  Ar/2(99) 

and 

^gt(^(«+I))  .g^(^(«))^t(«+l/2)(^(«))  (100) 

In  Cartesians,  this  scheme  reduces  to  the  usual 
Lorentz  force  leapfrog  integration  scheme  and  no 
iteration  is  required.  In  cylindrical  coordinates,  the 
predictor-corrector  can  be  replaced  by  direct  matrix 
inversion  in  the  manner  proposed  by  Boris  [7] .  Both 
Cartesian  and  curvilinear  coordinates  have  been  tried 
for  the  particle  integration,  and  for  orthogonal  and 
near  orthogonal  meshes  they  give  comparable  results. 
The  curvilinear  option  outlined  here  is  favoured  be¬ 
cause  it  requires  considerably  less  work  per  particle. 
However,  its  accuracy  on  distorted  elements  has  not 
yet  been  examined. 


3.4.6.  Particle  boundary  conditions 

Coordinates  of  particles  leaving  a  block  are  col¬ 
lected  and  sorted  into  tables  of  particle  passing  through 
given  glue-patches  or  boundary  condition  patches.  In¬ 
ternal  boundary  conditions  are  applied  at  the  glue- 
patches  by  transforming  the  particle  coordinates  to 
those  used  in  the  target  block.  External  boundary  con¬ 
ditions  are  applied  by  deleting  outgoing  particles  and 
adding  incoming  particles  according  to  the  surface  par¬ 
ticle  source  model  (e.g.  beam,  space  charge  limited 
emitter,  secondary  emitter,  etc) . 

3.5.  Coordinate  transformations 

The  coordinate  systems  used  in  the  software  are: 

(i)  global  Cartesians 

(ii)  block  Cartesians 

(iii)  block  orthogonal 

(iv)  block  contravariant 

(v)  block  CO  variant 

Global  positioning  information  comprises  two 
parts:  physical  position  and  logical  position.  The 
physical  position  of  a  block  is  described  in  terms 
of  its  origin  otooo  and  its  rotation  matrix  R.  Global 
physical  positioning  is  used  to  graphically  display  the 
data.  Logical  position,  described  by  the  glue-patches 
joining  blocks  together,  is  used  to  transform  curvilin¬ 
ear  field  and  particle  coordinate  components  between 
blocks.  This  is  described  in  Section  4.1.6. 

In  most  cases,  the  local  Cartesian  and  local  orthog¬ 
onal  coordinates  are  the  same.  However,  for  special 
cases  (e.g.  polar  mesh  segment  block) ,  it  becomes  ad¬ 
vantageous  for  metric  data  compression  to  store  the 
basis  vectors  e,  in  terms  of  components  of  a  reference 
orthogonal  system  which  is  not  Cartesian.  When  local 
orthogonal  coordinates  are  not  Cartesian,  their  scale 
factors  need  to  be  stored. 

The  basis  vectors  define  the  transformations  be¬ 
tween  orthogonal  coordinate  components  and  curvilin- 
ears,  and  the  covariant  /  contravariant  transformations 
use  the  metric  tensor.  The  orthogonal  components  are 
used  only  for  input  and  output. 

4.  Software 

The  management  of  complexity  has  been  the  major 
challenge  in  developing  the  PIC  software  suite  3DP1C; 
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Rg.  6,  A  schematic  of  the  division  of  the  3-D  electromagnetic  PIC  software  suite  3DPIC  into  preprocessor,  simulation  and  postprocessor 
program  units. 


to  produce  a  general  geometry  body-fitted  code  that 
is  both  portable  and  efficiently  utilises  multiprocessor 
distributed-memory  parallel  computers  has  required 
careful  hierarchical  structuring  of  both  data  and  pro¬ 
gram  units. 

The  first  division  is  into  the  interactive  (and  serial) 
programs  for  preprocessing  (PEGGIE  /  P3DTWT) 
and  postprocessing  (PICTIM  /  PICVIS),  and  into 
the  simulation  program  (PIC3D)  which  runs  non- 
interactively  on  the  parallel  computer.  This  is  shown 
schematically  in  Fig.  6.  The  preprocessors  take  com¬ 
pact  descriptions  of  the  device  geometry  and  material 
properties,  the  external  circuit  boundary  conditions, 
the  initial  values  for  the  simulation  and  the  selections 
of  output  datasets  to  be  generated  and  saved.  They 
validate  the  input  for  consistency,  and  produce  ini¬ 
tialisation  summaries  and  input  datasets  for  the  ker¬ 
nel  simulation  program.  Additionally,  data  may  come 
from  restart  files  from  previous  runs  and  from  play¬ 
back  files  (e.g.  beam  data)  from  other  software.  The 
preprocessors  exchange  some  flexibility  for  simplic¬ 
ity,  so  the  kernel  input  dataset  is  in  human  readable 
form  to  allow  the  insertion  of  special  conditions  not 
catered  for  by  the  preprocessors.  The  simulation  code 
PIC3D  produces  summary  output  and  binary  datasets 
for  the  postprocessors.  These  postprocessors  take  the 
time  series  and  snapshot  datasets  and  analyse  them 
to  produce  data  in  the  form  suitable  for  human  inter¬ 
pretation.  The  preprocessors,  simulation  program  and 
postprocessors  share  a  number  of  program  units,  and 
this  is  catered  for  by  grouping  the  program  units  into 
a  set  of  libraries.  A  discussion  of  the  multiblock  de¬ 


composition  is  presented  in  the  next  section,  followed 
by  a  brief  description  of  the  preprocessors  and  post¬ 
processors  in  Sections  4.2  and  4.3. 

4.1.  Multiblock  decomposition 

The  multiblock  decomposition  of  the  computational 
domain  into  a  set  of  curvilinear  hexahedral  bricks 
(uni blocks),  where  each  uniblock  has  its  own  fields 
and  particles  and  communicates  with  other  blocks  and 
boundaries  via  its  glue-patches,  provides  our  route  to 
efficiency,  portability  and  flexibility.  The  resulting  nu¬ 
merical  algorithm  within  each  block  is  almost  as  sim¬ 
ple  as  that  for  Cartesian  PIC  in  a  rectangular  brick.  The 
regular  (i,  j,  k)  addressing  within  each  block  allows 
data  compression  of  basis  vector  and  metric  storage 
(Section  4.1.5). 

Fig.  7  uses  the  simple  case  of  a  cylinder  to  illus¬ 
trate  the  multiblock  decomposition  of  a  complex  ob¬ 
ject  into  a  set  of  curvilinear  uniblocks  connected  by 
glue-patches.  The  cylinder  (a)  is  divided  into  flve 
blocks  (b).  These  blocks  are  treated  as  separate  ob¬ 
jects  with  their  own  coordinate  systems,  and  commu¬ 
nicate  by  passing  field  and  particle  data  via  the  glue- 
patches  (denoted  by  dashed  lines  in  (c)).  In  (d)  the 
curvilinear  quadrilateral  (hexahedral  in  3-D)  blocks 
are  meshed  by  transforming  them  to  rectangles  (bricks 
in  3-D)  in  curved  space  and  dividing  the  rectangles 
(bricks)  into  unit  side  square  (cube)  elements. 

The  simulation  program  treats  a  device  as  a  collec¬ 
tion  of  uniblocks  connected  together  by  a  network  of 
glue-patches.  Each  glue-patch  sticks  together  two  ad- 
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(a)  -  (b) 


Fig.  7.  An  illustration  of  the  multiblock  decomposition  and  mesh¬ 
ing  of  a  cylinder. 


jacent  block  faces  (or  block  edges  in  the  case  of  line 
patches) .  Different  device  are  constructed  by  glueing 
different  shaped  multiblocks  together  and  by  changing 
boundary  condition  patches  at  the  uniblock  surfaces. 
Different  material  types  are  treated  by  changing  the 
data  associated  with  the  block  type. 

Uniblocks  map  naturally  onto  processes  on  a  MIMD 
computer  architecture;  the  multiblock  decomposition 
may  be  thought  of  as  setting  up  a  set  of  independent 
PIC  calculations  on  each  processor,  with  the  (time  de¬ 
pendent)  particle  and  field  boundary  conditions  being 
provided  by  messages  from  the  other  processors.  All 
the  field  and  particle  data  for  a  given  block  is  local, 
so  the  only  data  that  must  be  exchanged  for  the  cal¬ 
culation  to  proceed  is  the  boundary  data. 

To  handle  complex  shapes,  some  uniblocks  will 
need  to  be  small,  whereas  for  simple  shapes,  physi¬ 
cal  boundary  condition  constraints  allow  large  blocks. 
If  desired,  large  blocks  can  be  subdivided  to  facilitate 
mapping  onto  the  MIMD  computer.  The  data  organi¬ 
sation  allows  several  small  blocks  to  be  collected  to¬ 
gether  on  a  given  process  to  obtain  effective  load  bal¬ 
ance  and  allows  uniblock  data  to  be  moved  from  one 
process  to  another. 

Without  compromising  the  subdivision  of  the  spa¬ 
tial  mesh  into  blocks  to  get  efficient  MIMD  process¬ 
ing,  one  can  further  demand  that  boundary  conditions 


Fig.  8.  The  data  in  the  main  simulation  program  PIC3D  is  divided 
into  global  addressing,  global  data  and  local  data. 


only  apply  at  the  surfaces  of  blocks  - .  This  completely 
eliminates  the  addressing  problems  caused  by  embed¬ 
ding  surfaces  within  blocks,  and  allows  surface  data 
to  be  passed  to  the  control  program  through  the  glue- 
patch  tables.  When  a  large  number  of  small  blocks  is 
needed  to  describe  a  complex  object,  load  balancing  is 
achieved  by  assigning  several  blocks  to  one  processor. 


4.1.1.  Global  addressing 

The  global  addressing  in  PIC3D  describes  the  dis¬ 
tribution  of  blocks  over  the  processors  of  the  MIMD 
computer,  and  the  logical  connection  of  the  blocks; 
this  addressing  is  used  to  pass  information  from  the 
surface  of  one  block  on  one  processor  to  the  surface 
of  another  block  which  may  be  on  another  processor. 
The  complete  global  addressing  network  can  be  de¬ 
scribed  by  the  positioning  of  the  two  faces  of  each 
glue-patch  on  the  uniblocks.  The  four  levels  in  this 
network,  the  processor,  process,  block  and  patch  are 
shown  schematically  in  Fig.  8. 

Processor:  To  aid  portability  a  distinction  is  made 
between  the  logical  processes  used  in  the  program  and 
the  physical  processors  of  the  MIMD  computer;  tables 
are  set  up  to  provide  the  mapping  between  processes 


2  This  condition  is  not  strictly  applied  in  the  program,  as  it 
was  found  advantageous  to  allow  special  blocks  with  embedded 
structures  to  describe  the  geometry  of  helical  travelling  wave  tubes. 
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and  processors.  The  simulation  program  is  written  in 
terms  of  processes,  which  are  translated  to  physical 
processor  numbers  by  these  tables.  When  the  software 
is  run  on  single  processor  workstations,  there  is  simply 
one  process  and  one  processor. 

Process:  Each  process  corresponds  to  a  running  ( mas¬ 
ter  or  slave)  copy  of  the  simulation  program,  and  may 
have  one  or  more  blocks  assigned  to  it.  This  allows  the 
computational  load  to  be  balanced  by  moving  blocks 
between  processes. 

Block:  To  each  block  corresponds  global  block  data, 
namely  the  block’s  location  in  space  relative  to  some 
global  coordinates,  its  type  and  global  attributes  of 
the  particles  within  it.  The  block  type  data  contains 
metric  information  needed  by  the  field  solver  and  by 
the  particle  integrator. 

Patch:  All  exchange  of  data  between  blocks  is  per¬ 
formed  through  the  patch  exchange  subroutine.  This 
recognises  whether  a  target  patch  to  which  it  is  to  send 
data  is  on  a  block  belonging  to  its  process,  or  on  a 
block  being  computed  by  another  process.  In  the  for¬ 
mer  case,  a  memory-to-memory  copy  of  the  data  is 
performed,  and  in  the  latter  a  message  is  sent  to  the 
target  process. 

Addressing  in  the  direction  of  the  downward  arrows 
in  Fig.  8  (processor  ->  process  block  ^  patch) 
uses  ordered  tables,  and  the  reverse  connections  use 
linked  lists. 

4.1.2.  Global  data 

Physical  and  numerical  data  which  spans  many 
blocks  is  stored  as  global  data;  examples  of  this  are 
particle  species  data,  boundary  condition  data,  block 
type  data  (e.g.  metrics)  and  global  Cartesian  posi¬ 
tions  of  the  blocks. 

4.1.3.  Local  addressing 

Local  addressing  manages  the  storage  of  the  prin¬ 
cipal  field  and  particle  arrays,  and  the  transfer  of  field 
and  particle  data  to  and  from  the  patch  buffers  used  in 
the  glue-patch  exchanges.  One-dimensional  address¬ 
ing  is  used  for  the  principal  field  and  particle  arrays 
to  allow  the  same  compact  coding  to  be  used  for  both 
2-D  and  3-D  computations.  Linked  lists  are  used  to 
sort  particles  into  the  glue-patch  buffers. 


4.1.4.  Local  data 

-  The  principal  local  data  arrays  are  those  for  nodal 
amplitudes  I',  d'  and  b',  for  the  particle  coordinates 
(5',  p'),  a  workspace  array  to  hold  alternately  E,-  and 
H;,  and  input  /  output  buffers  for  the  glue-patch  ex¬ 
changes.  On  a  given  process,  the  nodal  amplitude  data 
for  blocks  are  stored  as  a  sequence  of  fixed  length 
records  in  the  appropriate  field  or  current  array.  Parti¬ 
cle  and  buffer  data  use  variable  length  records. 

4.1.5.  Memory  usage 

PIC  codes  are  always  large  consumers  of  computer 
memory,  and  the  extra  data  needed  to  describe  gen¬ 
eral  geometry  can  greatly  inflate  this  need.  This  has 
been  avoided  in  PIC3D  by  using  data  compression  for 
metric  and  basis  vector  storage,  and  dynamic  memory 
allocation  for  the  large  local  data  arrays. 

In  most  cases,  symmetries  and  congruences  greatly 
reduce  the  amount  of  metric  data  to  be  stored.  For 
instance,  the  example  shown  in  Fig.  7  extended  to  3- 
D  perpendicular  to  the  plane  shown  has  two  block 
types  only;  the  central  rectangular  brick  block  type, 
which  requires  only  six  numbers  to  completely  specify 
its  basis  vectors  and  metric  tensor  elements,  and  the 
surrounding  wedge  shaped  block  type,  which  requires 
a  single  plane  of  values  for  metrics.  The  metric  data 
compression  for  this  case  has  reduced  the  memory 
requirement  from  the  full  metric  storage  for  the  block 
~  15N^  to  ~  7  -I-  3A",  assuming  dtn  N  x  N  x  N  mesh 
in  each  block.  These  savings  are  typical,  and  similarly 
apply  to  the  basis  vector  storage.  Physical  coordinates 
of  element  nodes  are  not  stored,  but  are  constructed 
when  needed  (for  diagnostic  output  only). 

The  preprocessors  compute  the  amount  of  memory 
required  for  the  field  arrays  and  estimate  how  much 
is  needed  for  particle  coordinates  and  patch  exchange 
buffers;  the  simulation  code  allocates  this  space  when 
it  starts  up. 

4.1.6.  Patch  exchange 

Patch  exchanges  are  used  to  transfer  field  and  par¬ 
ticle  data  between  blocks.  Field  glue-patch  exchanges 
correspond  to  the  transfer  of  fixed  length  records, 
whereas  the  length  of  records  for  particle  exchange 
varies  depending  on  the  particle  flux. 

Field  glue-patches.  For  all  cases,  the  application  of  in¬ 
terior  field  boundary  conditions  involves  assembling 
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glue-patch 


block  1 


block  2 


Fig,  9.  All  interior  boundary  conditions  are  applied  by  passing 
data  between  blocks  via  glue-patches. 


contributions  to  d  or  E  across  adjacent  block  bound¬ 
aries.  This  assembly  is  performed  by  adding  the  glue- 
patch  contributions.  Fig.  9  illustrates  the  glue-patch 
data  transfer.  Data  stored  on  the  logical  (i,  j,  k)  lattice 
of  nodes  in  block  1  are  copied  to  the  glue-patch,  and 
then  data  is  added  from  the  glue-patch  to  the  corre¬ 
sponding  nodes  on  block  2.  On  both  source  and  tar¬ 
get  blocks  are  “O’  and  “X”  reference  points  (solid 
circles  and  crosses  in  Fig.  9)  which  specify  the  loca¬ 
tion  and  orientation  of  the  blocks  with  respect  to  the 
glue-patch.  The  reference  points  are  used  to  perform 
the  field  component  and  indexing  transformations  be¬ 
tween  blocks. 

Two  types  of  field  glue-patch  are  required: 

-  surface  patches  to  exchange  the  two  tangential  field 
components  at  nodes  common  to  the  faces  of  two 
blocks; 

-  line  patches  to  exchange  the  one  tangential  field 
component  at  nodes  common  to  the  edges  of  four  or 
more  blocks.  If  a  node  is  common  to  M  blocks,  then 
it  has  associated  with  it  Af(M  -  3)/2  line  patches. 
Interior  boundary  conditions  are  applied  by  adding 

the  glue-patch  values  d^,„^  to  the  corresponding  node 
accumulator 

d'- :=  d' +  d';„,  (101) 

A  simple  2-D  example  of  the  use  of  glue-patches  is  to 
apply  doubly  periodic  boundary  conditions  to  a  rect¬ 
angular  domain.  Four  glue-patches  are  required:  two 
surfaces  patches  to  connect  the  north  to  south  bound¬ 
ary  and  the  east  to  west  boundary,  and  two  line  patches 
to  connect  the  NE  to  SW  corner  and  the  SE  to  NW 
corner.  Glue-patch  exchanges  are  required  twice  each 
timestep,  once  for  d'  to  complete  the  integration  of 
Ampere’s  equation,  and  once  for  E,  to  complete  the 


computation  of  E  from  d  using  the  constitutive  rela¬ 
tionship  (71). 

Particle  glue-patches.  Particle  exchange  requires  only 
surface  patches.  Each  field  surface  glue-patch  has  a 
corresponding  particle  glue-patch.  Particles  passing 
through  a  glue-patch  from  a  source  block  to  a  target 
block  are  passed  from  the  storage  areas  of  the  source 
block  particle  tables  to  that  of  the  target  block  via 
the  glue-patch  buffer.  Position  coordinates  are  trans¬ 
formed  from  the  curved  space  components  of  the 
source  block  to  those  of  the  target  block  by  using  the 
locations  of  the  “0“  and  “X”  keys  on  the  two  connect¬ 
ing  blocks  (cf.  Fig.  9) .  Particles  passing  through  more 
than  one  block  in  a  timestep  are  handled  by  repeated 
application  of  the  surface  glue-patch  exchange. 

4.2.  Data  input 

4.2.1.  Preprocessors 

The  generic  preprocessor  to  treat  general  device  ge¬ 
ometries  is  called  PEGGIE,  for  Parallel  Electromag¬ 
netic  Grid  Generator  Interface  and  Extensions.  As  im¬ 
plied  by  the  word  Interface,  PEGGIE  is  not  primarily 
intended  to  be  a  grid  generator,  although  it  has  power¬ 
ful  in-built  capabilities.  The  word  Extensions  relates 
to  the  ability  to  set  most  of  the  numerical,  physical 
and  housekeeping  variables  used  by  the  PIC3D  code. 

The  current  version  of  PEGGIE  accepts  command 
line  input.  The  use  of  a  small  number  of  tags,  com¬ 
bined  with  FORTRAN  90  NAMELIST  emulation  makes 
it  easy  to  specify  complex  objects  in  a  compact  way, 
set  control  and  diagnostic  parameters  in  a  consistent 
manner,  etc. 

The  in-built  grid  generator  uses  the  concept  that  the 
device  to  be  modelled  is  made  by  joining  together  sim¬ 
pler  objects  or  blocks,  each  of  which  is  cuboid  in  a 
suitably  chosen,  and  in  general  non-orthogonal,  coor¬ 
dinate  system.  Since  element  node  positions  are  deter¬ 
mined  by  transfinite  interpolation,  the  point  distribu¬ 
tion  along  three  adjacent  edges  defines  the  grid  within 
the  block.  If  the  points  are  distributed  uniformly  in 
some  (parametric)  sense,  then  three  integers  n,-,  i  = 
1,2,3  serve  to  define  the  local  mesh.  There  are  global 
constraints,  arising  from  the  need  for  adjacent  objects 
to  have  the  same  point  distribution  on  their  common 
boundary,  so  the  user  has  the  ability  to  suggest  n,  and 
let  PEGGIE  try  to  obtain  a  consistent  set. 
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The  algorithm  employed  is  most  easily  explained 
for  the  case  where  the  constituent  blocks  have  en¬ 
tire  faces  in  common  (although  partial  joins  are  sup¬ 
ported).  Essentially  a  trail  is  started  for  each  of  the 
three  adjacent  edges  that  define  a  block.  The  connec¬ 
tivity  or  patch  information  on  each  surface  implies  a 
meshing  in  two  coordinate  directions  for  the  adjacent 
block,  i.e.  two  trails  extend  into  it,  and  in  general,  an¬ 
other  trail  starts  there.  Successive  blocks  are  examined 
until  every  trail  has  ended,  typically  by  extending  from 
one  boundary  to  another.  The  trails  are  checked  for 
consistency  and  where  several  suggested  n;  lie  upon  a 
path,  the  one  chosen  is  that  corresponding  to  the  ear¬ 
liest  block  in  order  of  assembly. 

It  will  be  noted  that  blocks  may  have  different  orien¬ 
tations,  e.g.  the  1-  and  2-  coordinates  in  one  block  may 
correspond  to  respectively  the  3-  and  I-  coordinates 
in  an  adjacent  block.  Such  alignments  are  recorded 
by  use  of  integer  3  vectors  {a  b  c)  corresponding  to 
rotation  matrices 


where  Iq  is  a  unit  3-vector  with  one  entry,  the  a‘^, 
non-zero  and  having  the  same  sign  as  a,  e.g.  (1-32) 
corresponds  to  the  rotation  matrix 

Voi  0 ; 

A  similar  technique  has  been  devised  independently 
[24].  The  importance  of  the  {a  b  c)  notation  is  that 
quantities  such  as  the  n,-  may  be  transformed  simply 
by  permutation  and  sign  change. 

Once  blocks  have  been  gridded,  PEGGIE  proceeds 
to  generate  first  the  covariant  basis  vectors  e,  and  then 
the  coefficients  that  describe  the  combination  of 
block  geometry  and  (in  general)  tensor  permeability 
and  permittivity  as  explained  in  Section  3.3.4.  To  re¬ 
duce  computing  time,  block  symmetries  are  employed, 
so  for  example  in  an  orthogonal  cuboid  only  G(,)(/),  i 
=  1,  2,  3  for  one  cell  are  actually  calculated. 

The  Gy-  serve  to  set  a  maximum  timestep.  Provided 
e  and  jn  are  uniform,  isotropic  tensors,  it  may  rigor¬ 
ously  be  shown  for  a  grid  consisting  of  identical  par¬ 
allelograms,  that  a  necessary  and  sufficient  criterion 
for  stability  is 


This  contrasts  with  the  criterion  in  Ref.  [25]  that  has 
explicit  off-diagonal  terms,  apparently  because  the  4- 
point  sums  involving  off-diagonal  Gy  have  not  been 
fully  taken  into  account.  (The  off-diagonal  terms  ap¬ 
pear  implicitly  in  the  volume  element  g.)  In  3-D,  for 
general  but  still  homogeneous  G/y,  it  is  necessary  to 
resort  to  more  heuristic  arguments,  which  suggest  that 
stability  is  achieved  provided 


cAtx/r  <  — p 
v/3 


r  =  ^max(Gf,G".Gf2Gf„Gf,G3«. 

Gf3Gf„Gf2G3^3,Gf3G2''2)  (104) 

and  this  is  used  to  specify  At. 

PEGGIE  proceeds  to  calculate  line  patch  infor¬ 
mation,  needed  where  four  or  more  blocks  have  a 
common  edge.  An  important  feature  here  is  the  rep¬ 
resentation  of  a  block  as  the  cube  [-1,1]^  whose 
faces  may  be  labelled  by  3-vectors  {i\,i2,h)  = 
(±1,0,0),  (0,±1,0)  and  (0,0,±1)  according  their 
surface  normals.  Edges  can  be  represented  by  the  3- 
vectors  that  are  the  sums  of  two  adjacent  face  vectors, 
and  corners  similarly  as  sums  over  the  three  faces 
that  meet.  These  3-vectors  can  each  be  condensed  to 
one  integer  i  =  6iy  ±  2/2  +  i\  and  the  i  values  provide 
a  natural  ordering  for  respectively  faces,  edges  and 
corners,  e.g.  DSWENU  for  faces  where  D,  S,  W,  E,  N 
and  U  represent  the  Down,  South,  West,  East,  North 
and  Up  faces  of  the  cube  [  —  1,1]^  ( so  D  corresponds 
to  the  face  with  normal  (0,  0,  -1),  etc).  A  single  in¬ 
teger  operation  decides  which  is  the  other  cube  face 
that  shares  a  specified  edge  with  a  given  face. 

Answering  the  latter  question  is  the  basis  of  the  line 
patch  calculation:  the  need  is,  for  each  a  block  edge, 
to  find  a  list  of  blocks  which  are  coincident  along  at 
least  part  of  that  line,  given  only  surface  connectivity 
information.  The  idea  is  to  describe  a  trail  about  the 
edge  through  the  adjacent  blocks,  which  may  be  a 
closed  path,  or  may  end  at  device  boundaries,  or  even 
in  general  may  split.  The  first  stage  of  the  algorithm 
is  to  rearrange  the  surface  patch  information,  so  in 
particular  that  for  every  edge  of  every  block,  there  is 
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a  list  of  contiguous  surface  patches.  Starting  with  an 
unmarked  edge  a  trail  is  pursued,  marking  each  block 
edge  (including  the  starting  edge)  that  is  found  to 
be  coincident  with  the  first  one,  until  it  terminates  as 
described  above,  or  splits,  in  which  case  two  separate 
trails  are  begun  and  pursued. 

The  last  large  set  of  non-local  quantities  to  be  com¬ 
puted  is  of  the  positions  ( xooo)  of  the  blocks  and  their 
orientations  (R)  which  respect  to  a  global  coordinate 
system.  The  position  and  orientation  of  one  specified 
block  defines  this  frame. 

At  this  point  PEGGIE  calculates  the  scalings  re¬ 
quired  to  convert  fields  to  the  dimensionless  units  used 
by  PIC3D.  These  are  used  at  once  in  the  generation 
of  coefficients  that  treat  boundary  conditions  of  an 
applied  electric  field,  resistive  wall  etc  (see  Section 
3.3.6). 

The  timestep  and  scaled  block  dimensions  are  also 
employed  to  define  control  parameters;  if  the  length  of 
a  P1C3D  run  is  specified  by  the  number  of  timesteps, 
PEGGIE  works  out  suitable  times  for  the  saving  of 
diagnostic  information  to  disc.  Line,  surface  and  vol¬ 
ume  integral  diagnostics  (cf.  Section  4.3)  may  be  de¬ 
manded,  that  may  extend  over  different  blocks. 

Making  full  use  of  the  compressed  storage  tech¬ 
nique  (Section  4.1.5),  scalings  are  applied  to  e,  and 
Gp.  Finally  the  remaining  tables,  such  as  the  block 
to  process  mapping,  (essential  for  PIC3D  operations) 
are  set,  then  the  kernel  data  file  suitable  for  immediate 
input  to  PIC3D  is  written  to  disc. 


4.2.2.  TWT preprocessor 

The  travelling  wave  tube  (TWT)  preprocessor  is 
the  first  of  a  family  of  preprocessors  which  are  de¬ 
signed  for  specialist  application  areas.  It  builds  on  ex¬ 
isting  experience  of  modelling  TWTs  in  2-D  [26]. 
This  preprocessor,  which  calls  upon  the  library  of  rou¬ 
tines  developed  for  PEGGIE,  provides  a  much  simpler 
but  more  restricted  method  of  initialisation. 

The  preprocessor  was  developed  in  collaboration 
with  TWT  tube  engineers  from  EEV  Ltd,  and  has  been 
designed  to  take  parameters  with  which  the  tube  en¬ 
gineers  prefer  to  work  and  generates  the  detailed  in¬ 
put  description  that  the  main  simulation  code  requires. 
The  engineer  is  not  required  to  set  any  parameters  to 
define  the  multiblock  and  element  subdivision  of  the 
tube  interaction  space,  or  the  number  of  superparticles 


Fig.  10.  A  preprocessor  output  showing  a  section  of  the  blocking 
of  a  TWT.  Shown  are  a  central  cylindrical  block  containing  three 
turns  of  the  helix,  surrounded  by  three  further  blocks.  The  thicker 
lines  are  block  boundaries,  and  the  thinner  are  helix  element 
boundaries. 

to  be  used,  or  any  other  numerical  parameter  to  be 
used,  although  they  can  override  the  defaults  if  they 
wish. 

The  user  input  file  ( .  uif )  used  by  the  TWT  pre¬ 
processor  uses  the  FORTRAN  90  NAMELIST  format. 
An  input  file  can  include  other  files  to  allow  users  to 
set  their  own  defaults,  allowing  a  series  of  related  cal¬ 
culations  to  be  performed  with  the  minimum  amount 
of  input.  One  important  feature  of  the  preprocessor 
is  the  ability  to  produce  validation  output  to  confirm 
that  the  main  simulation  code  is  correctly  initialised, 
an  example  of  one  such  output  is  shown  in  Fig.  10. 

4.3.  Data  output 

The  main  simulation  code  uses  a  hierarchical  ap¬ 
proach  for  specifying  the  output  required;  plot  sets  de¬ 
fine  the  output  file  format,  the  field  set,  the  domain  set 
and  the  time  set.  The  field  set  specifies  a  set  of  quan¬ 
tities  to  be  plotted  for  each  domain  in  the  domain  set, 
at  the  times  listed  in  the  time  set.  Each  domain  is  a 
union  of  subdomains,  each  of  which  belong  to  a  single 
block.  The  implementation  is  designed  to  allow  new 
operations  and  output  formats  to  be  added  easily. 

The  present  version  of  the  software  has  snapshot 
output  for  a  limited  range  of  geometries,  and  time- 
series  output  field  specifications  for  point  quantities, 
line  integrals  (e.g.  /  E-dl  or  f  H -dl),  surface  inte- 
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grals  if  D-  dS,  f  B  •  dS  or  f  j  •  dS)  and  volume  in¬ 
tegrals  ( /  ipdV  where  ift  may  be  for  example  particle 
number,  B  ■  H/2,  E  •  D/2  or  j  ■  E). 

The  implementation  of  the  diagnostic  output  pro¬ 
duces  separate  output  for  each  process.  Post  proces¬ 
sors  can  then  combine  data  from  several  processes  into 
a  single  dataset.  This  is  satisfactory  provided  that  the 
subdomains  of  a  diagnostic  domain  set  reside  on  the 
same  process,  but  for  more  general  situations  message 
passing  will  also  be  needed  for  diagnostics. 

5.  Parallel  benchmarking 

The  message  passing  model  used  in  PIC3D  matches 
the  structure  of  massively  parallel  processors  (MPPs) 
such  as  the  Intel  Paragon,  IBM  SP/2  and  Meiko  CS-2, 
and  is  also  implemented  efficiently  on  the  parallel  vec¬ 
tor  shared-memory  computers  such  as  the  Cray  Y-MP 
and  C-90.  PiC3D  currently  uses  the  Intel  NX/2  library 
of  communication  subroutines  (mostly  CSEND  and 
CRECV)  because  these  are  implemented  with  min¬ 
imum  overhead  on  many  computers  and  are  widely 
used.  A  version  of  the  code  using  the  popular  PVM 
message  passing  language  [27]  has  been  written,  and 
tested  on  an  IBM  SP/2.  This  gives  maximum  porta¬ 
bility,  but  the  scaling  with  increasing  number  of  pro¬ 
cessors  is  unsatisfactory  if  Ethernet  and  the  the  Inter¬ 
net  Protocol  (IP)  are  used,  due  to  their  high  message 
latency.  However  the  PVM  code  is  expected  to  scale 
satisfactorily  when  run  under  IBM’s  customized  im¬ 
plementation  PVMe  [28]. 

Both  2-D  and  3-D  test  cases  have  been  used  to  eval¬ 
uate  parallel  performance.  The  former,  LPM2  bench¬ 
mark  uses  the  realistic  device  geometry  of  a  magneti¬ 
cally  insulated  line  oscillator  (MILO),  and  the  latter, 
LPM3  benchmark  uses  a  3-D  periodic  plasma  to  pro¬ 
vide  a  clearly  scalable  test  which  does  not  depend  on 
non-reproducible  details  such  as  random  number  gen¬ 
erators. 

5.1.  LPM2  benchmarking 

The  choice  of  a  2-D  MILO  case  for  LPM2  was 
made  because  the  geometry  reflects  realistic  engineer¬ 
ing  calculation  requirements  and  the  results  could  be 
compared  with  those  from  other  PIC  codes  [29,30]. 
Fig.  1 1  shows  the  electron  scatter  plot  for  a  20  block 
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Fig.  11.  An  example  of  snapshot  output  of  2-D  MILO  simulation 
by  PIC3D.  This  case  is  divided  into  20  blocks;  9  cavities  and  11 
interaction  space  blocks. 


subdivision  of  a  MILO  simulated  using  PIC3D  in  its 
2-D  mode. 

LPM2  benchmarking  runs  undertaken  on  the  Intel 
iPSC  for  a  64  block  decomposition  of  a  MILO,  using 
a  total  of  approximately  10,000  elements  and  13,000 
particles  showed  better  than  eighty  percent  of  the  the¬ 
oretical  speed-up  for  up  to  16  processors,  despite  the 
non-uniform  distribution  of  particles  and  small  prob¬ 
lem  size.  However,  beyond  this  limit  (i.e.  <  4  blocks 
per  process)  the  number  of  timesteps  per  second  be¬ 
gan  to  saturate  (clearly,  beyond  64  processors  further 
speed-up  is  impossible  without  further  problem  sub¬ 
division!). 

5.2.  LPM3  benchmarking 

The  three-dimensional  periodic  plasma  test  case 
LPM3  (Local  Particle  Mesh  benchmark  #3)  was  run 
on  the  Intel  Paragon  and  Intel  iPSC/860.  This  bench¬ 
mark  problem  was  chosen  to  be  a  representative  test 
of  the  features  of  PIC3D  on  the  available  MPPs,  to  be 
easily  scalable  in  size  and  to  be  physically  realistic. 
LPM3  explores  the  capabilities  of  MPPs  with  more 
than  a  thousand  processors. 

In  LPM3,  the  plasma  space  is  divided  into  blocks, 
each  of  which  consists  of  512  particles  representing 
the  electrons,  and  64  elements  on  which  the  fields  are 
calculated.  From  the  point  of  view  of  load  balanc¬ 
ing  the  parallel  computer,  the  block  is  the  smallest 
unit  that  can  be  allocated  amongst  the  processors.  The 
problem  size  is  measured  by  the  number  of  blocks  A*, 
and  four  problem  sizes  have  been  used  with  respec¬ 
tively  Nb  =  8, 64, 512, 4096.  These  correspond  respec¬ 
tively  to  numbers  of  particles  Np  =  AK,  32K,  256K, 
2M2  where  K  =  1024  and  M2  =  K^.  The  timestep  is 
such  that  about  ten  percent  of  the  particles  transit  to 
neighbouring  blocks  during  a  timestep.  A  run  of  100 
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timesteps  is  chosen  as  the  benchmark  test  because  this 
takes  only  a  few  minutes  for  the  problem  sizes  and 
number  of  processors  of  interest,  and  the  conservation 
of  total  number  of  particles  is  used  as  a  validity  check. 

There  are  two  different  versions  of  the  benchmark 
code,  which  are  selected  by  an  input  variable.  The  per- 
patch  version  sends  a  separate  message  for  every  patch 
in  the  system,  and  there  are  9  patches  for  every  block. 
In  the  per-process  version,  on  the  other  hand,  the  patch 
messages  are  assembled  and  sorted  in  a  buffer  so  that 
only  one  message  is  sent  to  every  other  process  to 
which  a  given  process  is  attached.  The  per-process 
code  may  send  10  or  100  times  fewer  messages  than 
the  per-patch  version,  and  so  should  be  significantly 
faster  than  the  per-patch  version  on  computers  with  a 
high  message  start-up  time  or  latency.  This  difference 
is  clearly  seen  when  the  code  is  run  on  the  IBM  SP/2 
using  the  high  latency  PVM-IP  message  passing  in¬ 
terface.  For  computers  with  low  latency  such  as  the 
Intel  Paragon  using  NX/2  under  SUNMOS,  there  is 
little  difference  between  the  versions. 

Results:  All  the  benchmark  measurements  obtained 
are  shown  in  Fig.  12.  The  results  are  expressed  as 
temporal  performance  Rp  in  units  of  timestep  per  sec¬ 
ond  {tstep  an  unambiguous  metric  which  ex¬ 

presses  exactly  what  a  program  user  needs  to  max¬ 
imise  [31].  We  have  deliberately  not  used  the  some¬ 
times  popular  metrics  of  Parallel  Speed-up  and  Effi¬ 
ciency  because  these  are  not  absolute  measures  and 
cannot  be  used  correctly  to  compare  the  performance 
of  different  computers  [31-33].  Neither  have  we  ex¬ 
pressed  the  results  as  Megaflop  per  second,  because 
this  would  make  the  curves  lie  closely  on  top  of  each 
other,  and  disguise  the  actual  time  of  computation  from 
the  reader.  In  our  units  of  timestep  per  second,  calcu¬ 
lations  which  take  the  same  time  lie  at  the  same  height 
in  the  graph;  faster  calculations  lie  higher  and  slower 
lie  lower.  These  statements  would  not  necessarily  be 
true  if  the  results  were  expressed  as  speed-up  or  in 
megaflop  per  second. 

For  each  block  size  there  are  two  pairs  of  curves. 
The  pair  closest  to  the  theoretical  dotted  line  is  that 
for  the  Intel  Paragon  running  under  the  Sandia  SUN¬ 
MOS  1.4.8  operating  system,  and  the  pair  about  60 
percent  lower  and  slower  are  for  the  iPSC/860  run¬ 
ning  under  the  NX  release  3.3.2  operating  system.  For 
each  computer  the  open  symbols  are  the  measured  val- 


Number  of  Processors,  p 

Fig.  12.  Temporal  Performance  of  the  LP.M3  benchmark  measured 
in  units  of  timestep  per  second,  for  four  problem  sizes  and  up  to 
1840  processors  on  the  Sandia  Laborator>’s  Intel  Paragon  XP/S 
140  (upper  pair  of  curves  which  are  close  to  the  dotted  lines), 
and  up  to  64  processors  on  the  Intel  iPSC/860  (lower  pair  of 
curves).  See  text  for  further  details. 

ues  using  the  per-process  code  and  the  corresponding 
filled  symbols  for  the  per-patch  code.  The  fact  that  the 
curves  for  both  versions  are  almost  the  same  on  both 
the  computers,  means  that  message  latency  is  not  a 
problem  with  the  Paragon  under  SUNMOS  or  with  the 
iPSC/860.  The  latency  has  separately  been  measured 
using  the  COMMSl  “pingpong’  benchmark  [33]  to 
be  about  80  /zs  for  both  these  computers. 

The  dotted  lines  are  the  theoretically  perfect  linear 
scaling  predictions  for  the  Paragon,  calculated  from 
the  one-processor  performance  on  the  eight  block 
problem.  The  theoretical  scaling  assumes  that  perfor¬ 
mance  is  proportional  to  the  number  of  processors, 
and  inversely  proportional  to  the  problem  size;  ide¬ 
ally,  p-times  as  many  processors  should  compute  an 
existing  problem  p-times  faster  (the  speed-up),  or  a 
problem  p-times  bigger  in  the  same  time  (the  size- 
up).  Alternatively,  of  course,  the  increase  in  number 
of  processors  should  be  able  to  be  used  to  obtain  a 
combination  of  speed-up  and  size-up.  The  stepwise 
nature  of  the  measured  performance  curves  arises  due 
to  load  imbalance  when  the  number  of  blocks  is  not 
exactly  divisible  by  the  number  of  processors.  If  we 
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concentrate  attention  on  the  best  performance  figures 
which  correspond  to  perfect  load  balance,  when  every 
processor  is  computing  the  same  number  of  blocks, 
we  find  that  the  measured  performance  is  within  80 
percent  of  the  ideal  linear  scaling  except  for  the  512 
block  case  with  greater  than  256  processors,  and  for 
the  4096  block  case  with  more  than  1366  processors. 
For  these  cases  with  larger  numbers  of  processors, 
performance  saturation  is  beginning  to  be  seen. 

For  the  set  of  cases  shown  in  Fig.  12  it  is  evident 
that  there  is  not  enough  parallelism  in  the  problem 
to  obtain  more  than  about  10  tstep  j“'.  The  8  block 
problem  can  at  most  use  8  processors,  and  there  is  no 
possibility  of  using  more  processors  to  speed  up  that 
problem.  The  best  we  can  do  with  the  extra  processors 
is  to  solve  bigger  problems  in  about  the  same  time, 
and  this  is  seen  for  the  cases  of  Ni,  =  64,  512,  4096, 
none  of  which  exceed  10  tstep  however  many 
processors  are  used. 


6.  Final  remarks 

We  have  devised  and  implemented  new  algorithms 
for  electromagnetic  Particle-in-Cell  (PIC)  calcula¬ 
tions  to  enable  the  simulation  of  realistic  devices 
with  complex  geometry.  These  algorithms  build  on 
existing  experience  with  the  electromagnetic  PIC  ap¬ 
proach;  indeed  in  simple  geometries  they  reduce  to 
state-of-the-art  algorithms.  They  are  economical  in 
that  under  such  circumstances  they  cost  no  more, 
and  in  that  they  perform  efficiently  on  virtually  all 
types  of  computer,  i.e.  on  single  and  multiple  proces¬ 
sor  machines  which  may  or  may  not  share  memory. 
Moreover  many  features  of  the  algorithms  serve  to 
reduce  software  costs,  by  making  it  easy  to  change 
the  properties  of  a  model  by,  say,  altering  its  dielectric 
properties  or  surface  material. 

More  precisely,  our  algorithms  extend  the  finite  el¬ 
ement  method  of  Lewis  [  16]  to  more  general  element 
shapes  and  to  the  time  variable.  These  methods  avoid 
the  quadrature  problems  encountered  in  earlier  gener¬ 
alisations  [34],  whilst  automatically  ensuring  charge 
conservation.  The  variational  formulation  gives  algo¬ 
rithms  with  the  “energy  conserving”  rather  than  the 
“momentum  conserving”  property.  If  desired,  similar 
momentum  conserving  schemes  can  be  obtained  in 
general  geometry  without  losing  the  charge  conserva¬ 


tion  property  by  introducing  a  field  reconstruction  step 
before  interpolating  force  to  particle  positions. 

We  have  focused  in  this  paper  on  elements  which 
transform  to  unit  cubes  in  some  general  coordinate 
system;  these  were  chosen  for  reasons  of  elegance  and 
computational  efficiency.  However  elements  of  higher 
order  and  of  different  shape  could  be  treated.  With 
cuboid  elements,  the  tensor  formulation  provides  sim¬ 
ple  expressions,  differing  little  from  those  obtained 
using  finite  differences  on  a  Cartesian  mesh,  for  the 
update  of  fields  and  particle  motion,  yet  allows  fully 
body-fitted  subdivision  of  the  computational  domain. 
The  problems  of  locating  the  element  in  which  a  par¬ 
ticle  lies,  of  computing  its  acceleration  and  associated 
current,  and  of  integrating  Maxwell’s  equations,  are 
no  more  difficult  than  in  Cartesian  PIC. 

The  form  of  the  numerical  algorithm  allows  data  to 
be  organised  straightforwardly  into  a  form  suitable  for 
distributed-memory  parallel  computers.  This  has  made 
it  possible  for  3DPIC  to  be  coded  in  FORTRAN  77, 
rather  than  have  to  submit  to  the  stricter  programming 
regime  of  C-f- 1-  and  endure  its  shortcomings  regarding 
particularly  speed  of  execution  [35]. 

The  physics  of  Maxwell’s  equations  provides  a  nat¬ 
ural  organisation  of  the  data,  in  that  interactions  at  a 
point  in  space  only  involve  information  from  its  light 
cone  in  space-time.  Our  software  uses  this  fact  to  di¬ 
vide  a  large  PIC  computations  into  many  smaller  ones 
which  communicate  only  with  their  immediate  neigh¬ 
bour.  Hence  work  can  be  shared  evenly  amongst  the 
processors  of  a  MIMD  machine,  whilst  minimising 
the  amount  of  global  data  and  interprocessor  message 
passing.  In  addition,  the  simple  logical  cube  address¬ 
ing  within  each  block  leads  to  fast  serial  processing 
within  each  slave  process. 

Benchmarking  confirms  that  our  multiblock  data 
organisation  offers  large  computational  intensity  and 
weak  Amdahl  limit  on  parallel  processor  speed-up 
[  14].  In  particular,  we  have  shown  that  doubling  the 
size  of  the  computer  allows  simulations  of  twice  the 
size  to  be  undertaken  in  the  same  elapse  time.  A  sim¬ 
ilar  approach  [36]  to  dividing  the  computational  do¬ 
main  has  demonstrated  very  high  efficiencies  for  PIC 
calculations  in  simple  geometries. 

An  alternative  to  the  hierarchical  multiblock  ap¬ 
proach  is  to  use  a  completely  unstructured  mesh  [37- 
41  ] .  This  has  the  advantage  that  topologically  difficult 
objects  can  be  more  readily  meshed,  but  the  disadvan- 
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tages  that  the  large  data  compressions  and  the  ease  - 
with  which  particles  can  be  tracked  in  our  multiblock 
method  are  not  readily  attainable. 

The  preprocessor,  simulation  and  postprocessor 
software  packages  described  in  this  paper  are  not 
yet  fully  mature,  but  are  sufficiently  well  advanced 
to  show  that  they  will  provide  effective  tools  for 
computer  simulation  studies  of  not  only  parts  of  mi¬ 
crowave  devices,  but  given  suitable  MPPs,  of  entire 
microwave  systems  with  hitherto  inaccessible  scale 
lengths. 
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G  M  AUI-3DPIC /README-MIMD 


readme-mimd 


File:  -roger/MAUI-3DPIC/README-MIMD 


MIMD  VERSION  PIC3D 


OPERATING  INSTRUCTIONS 


MAUI  HPCC 


Roger  W.  Hockney 
24  August  1995 


1)  PVM3  PARALLEL  VERSION 


The  MIMD  version  of  PIC3D  has  initially  been  set  up  under  public 
domain  PVM3 .  It  is  recognised  that  this  is  not  the  most  efficient 
implementation  of  message  passing  or  of  PVM,  but  PVM3  is  widely 
available  and  this  version  should  be  easily  portable.  Experience 
on  the  Maui  facility  is  that  PVM3  is  the  most  robust  message 
passing  system  and  does  not  need  dedicated  nodes.  It  is  thus  the 
easiest  to  use  for  test  purposes.  Message  passing  interfaces  with 
significantly  less  message  latency  are  IBM's  native  message 
passing  library  (MPL)  and  IBM's  efficient  implementation  of  PVM 
(called  PVMe) .  Conversion  to  MPL  and  PVMe  is  considered  a  second 
stage  and  should  not  be  too  difficult,  however  both  systems  need 
dedicated  nodes  on  the  SP2  which  can  be  difficult  to  get.  Testing 
of  these  versions  will  therefore  be  slower  than  PVM3 . 

2)  LOGING  IN  TO  MAUI 


In  order  to  login  to  Maui  from  Culham,  do  the  following: 

%  is  the  Unix  prompt  at  both  Culham  and  Maui, 

<PWD>  is  the  password,  and 
.  .  .  means  some  lines  are  omitted 

rfesunc%  xterm&  Start  an  'xterm'  window,  otherwise  you  will 

not  be  able  to  edit  with  'vi'  properly. 

I  usually  open  three  xterm  windows,  one  for  editing  and  recompilation, 
and  two  following  the  Unix  processes  on  two  of  the  nodes  forming  the 
PVM  computer  using  the  'ps'  command.  The  next  login  procedure  must  be 
done  in  each  window.  The  node  allocated  to  your  login  is  chosen  by 
the  system,  but  afterwards  you  can  'rlogin'  to  another  node. 

The  full  name  for  each  node  (there  are  400  of  them)  is  of  the  form 

fr<frame  number>n<node  number> .mhpcc . edu 

Thus  'fr8n09.mhpcc.edu'  is  node  9  of  frame  8 
This  full  name  is  usually  necessary  if  it  is  requested. 

A  frame  either  holds  16  'thin'  nodes  numbered  1 , 2 , 3 , . . . , 16,  or 
8  'fat'  nodes  numbered  1 , 3 , 5, . . . , 15 . 

The  full  Internet  name  of  the  Maui  SP2  is: 

tsunami . sp2 .mhpcc . edu  (164.122.163.114) 

A  'tsunami'  is  evidently  one  of  those  huge  tidal  waves  that  hit  the 
islands  from  time  to  time.  Maui  HPCC  was  shut  down  for  a  day  last 
year  because  of  a  real  one  of  these  being  forecast. 

This  is  the  login  procedure: 

rfeEunc%  telnet  gw  In  the  xterm  window,  telnet  through  the 

Harwell  gateway  computer 

gw  telnet  proxi  (Version  VI. 3)  ready: 

Username:  rhoc)cney 
Password:  <PWD> 

Login  Accepted 

tn-gw->  c  tsunami.sp2.mhpcc.edu 
Trying  164.122.163.114  port  23... 

telnet  ( fr?n?? .mhpcc . edu)  The  system  tells  you  which  frame 

and  node  number  has  accepted  your 
login. 
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AIX  Version  3 
(C)  Copyrights  .  .  . 
login:  hockney 
hockney's  Password:  <PWD> 


Last  login:  ... 

%  Is  Unix  prompt  from  Maui  SP2 ,  then  'Is' 

The  following  directories  concern  us: 

MAUI-3DPIC  MIMD  PVM3  version  of  PIC3D,  subdirectories 

below  this  copy  those  at  Culham. 

SP2  LPM3  benchmark  code  used  for  testing 

message-passing  interface. 

pvm3/bin/RS6K  Directory  for  binary  executables,  expected 

by  PVM3 .  The  pic3d  executable  MUST  be 
copied  to  here  after  any  recompilation. 

%  hostname  Type  at  any  time  to  find  out  your  node's 

fr8n08.mhpcc.edu  name. 

3)  SETTING  UP  PVM3 


It  is  first  necessary  to  set  up  a  PVM3  parallel  machine  (here  called 
your  PVM)  from  the  available  nodes  of  the  IBM  SP2 .  Perform  the  following 
steps  on  the  node  of  the  SP2  that  has  accepted  your  login,  here  assumed 
to  be  node  fr8n08.mhpcc.edu.  For  example  to  get  a  parallel  machine  with 
2  nodes : 

%  pvm  Start  PVM3  daemon  on  your  login  node,  and  also 

start  the  PVM  console  on  the  same  node. 

The  console  is  used  to  control  the  PVM, 
add  and  remove  nodes,  test  status  of  the  PVM, 
etc.  The  console's  prompt  is  'pvm>',  and  the 
available  commands  are  given  by  typing  'help' . 

pvm>  conf  List  the  configuration  of  your  PVM. 

1  hosts,  1  data  format 

HOST  DTID  ARCH  SPEED 

fr8n08.mhpcc.edu  40000  RS6K  1000 

It  comprises  just  the  one  node  that  we  logged  on  to. 

Now  we  add  1  more  node: 

pvm>  add  fr8n09.mhpcc.edu 

1  successful 

HOST  DTID 

fr8n09.mhpcc.edu  100000 

This  starts  a  PVM3  daemon  on  the  second  node  and  thereby  adds  it  to 
your  PVM  configuration.  To  confirm  that  we  have  a  PVM  configuration 
of  two  nodes,  type  'conf': 

pvm>  conf  List  the  configuration  of  your  PVM. 

2  hosts,  1  data  format 

HOST  DTID  ARCH  SPEED 

fr8n08.mhpcc.edu  40000  RS6K  1000 

fr8n09.mhpcc.edu  100000  RS6K  1000 

pvm>  quit  Quit  the  PVM  console, 

pvmd  still  running  but  leave  PVM  running,  i.e.  your  PVM  computer 
is  there  but  you  just  do  not  have  its  console 
running.  The  console  can  be  restarted  at  any  time 
by  typing  'pvm'  to  the  Unix  prompt  of  any  node 
that  is  part  of  the  PVM. 

Your  PVM  is  now  setup,  and  a  parallel  code  can  be  started  by  executing 
one  copy  on  either  of  the  two  nodes  of  the  PVM.  The  copy  started  will 
detect  that  it  is  the  first  or  'master'  copy  and  whether  it  needs  to 
spawn  off  other  copies  of  the  code  on  other  nodes  to  make  up  the  number 
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of  processes  NPRES  specified  in  the  data  file.  If  NPRES  >  1  these  extra 
processes  are  spawned  and  started.  The  parallel  code  is  then  running. 
Remember  that  PVM  takes  the  executable  to  be  used  in  the  spawned 
copies  from  the  directory  -/pvm3/bin/RS6K,  so  after  a  recompilation  the 
executable  must  be  copied  to  this  directory  if  you  want  it  to  be  used  by 
PVM!  There  can  be  much  confusion  if  you  forget  to  do  this. 

The  processes  are  assigned  to  the  nodes  of  your  PVM  in  the  sequence  given 
by  the  'conf'  command,  so  that  if  the  total  number  of  processes  equals 
the  number  of  nodes  of  the  PVM  then  there  will  be  one  process  per  node, 
as  desired.  If  there  are  more  processes  than  nodes,  the  nodes  are  reused 
in  a  cyclic  fashion,  and  there  will  be  more  than  one  PIC3D  process 
executing  on  each  PVM  node.  This  is  OK  because  each  PIC3D  process  is 
implemented  as  a  separate  Unix  process  which  can  be  seen  if  you  login 
to  a  node  and  do  a  'ps',  e.g: 

%  ps  -elf  I  grep  hockney 

In  reply,  the  PVM  daemon  that  runs  on  every  node  of  the  PVM  and  holds  the 
PVM  together ,  shows  up  as : 

.  .  .  /source/pd/pvm3 .2 . 6/pvm3/lib/RS6K/p 

and,  if  running,  the  console  shows  up  as  an  additional  process 

.  .  .  pvm 

The  PIC3D  process  shows  up  as 

.  .  .  .  .  .  pic3d 

In  fact,  if  you  execute  the  PVM  code  with  NPRES=128  on  a  PVM  of  one  node 
you  will  find  128  copies  of  the  executable  'pic3d'  running  on  the  node. 

If  the  PVM  console  fails  to  add  nodes  when  requested,  or  fails  to  start, 
do  a  'ps'  and  remove  any  old  PVM  daemons  that  might  have  been  left  from 
previous  sessions  that  were  not  ended  cleanly.  Also  go  to  /tmp  and  remove 
any  files  belonging  to  'hockney'  and  called: 

/tmp/pvmd. 544 
/tmp/pvml . 544 

Don't  ask,  just  do  it! 

To  end  a  PVM  session  cleanly,  invoke  the  PVM  console,  and  type  'halt'; 

%  pvm 

pvmd  already  running. 
pvm>  halt 

libpvm  (t40002]:  mxfer()  EOF  on  pvmd  sock 

That  should  avoid  many  problems  when  you  try  to  setup  a  PVM  next  time. 

When  you  setup  a  PVM  computer  in  the  above  way,  it  is  well  to  remember 
that  your  PVM  may  overlap  many  other  users  PVMs,  and  when  you  do  a 
'ps  -elf'  you  may  see  many  other  PVM  daemons  belongong  to  other  users. 
This  allows  many  parallel  programs  to  be  tested  by  many  users  without 
requiring  a  set  of  dedicated  nodes  to  be  assigned  to  each  user.  It  also 
means  however  that  your  PVM  is  time-sharing  the  CPU  of  each  node  with 
other  users,  and  therefore  no  meaning  can  be  attached  to  the  execution 
times  that  are  recorded.  The  only  way  to  do  proper  timing  is  to  make 
application  for  the  use  of  nodes  dedicated  to  your  application  alone. 

All  the  above  is  a  bit  tedious,  but  is  how  public-domain  PVMS  is  used. 
Many  computer  manufacturers'  versions  of  PVM3  are  simpler,  and  therefore 
probably  better,  in  that  they  do  not  allow  the  PVM3  spawning  instruction. 
Their  own  operating  system  takes  over  the  responsibility  of  providing 
the  required  number  of  nodes,  and  the  users  executable  is  started  on 
each  of  these  nodes.  The  Cray  T3D  and  Meiko  CS2  implementations  are  like 
that.  However  here  we  have  described  the  portable  public-domain  PVM3  from 
Oakridge  as  is  available  free  by  anonymous  ftp  or  from  the  Web,  and 
commonly  used  on  workstation  clusters. 


4)  DIRECTORY  STRUCTURE  AND  TEST  CASES 


4 . 1  The  Executable 
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The  executable,  'pic3d' ,  for  the  parallel  version  of  PIC3D  is  in 
directory: 

-hockney/MAUI-3DPIC/PIC3D 

and  copied  for  use  by  the  PVM3  spawn  subroutine  into: 

-hockney/pvm3 /bin/RS6K 
4.2  Test  Data  and  Results 


The  data  and  results  for  various  test  cases  are  in  directories,  which 
are  repeated  also  in  the  Culham  directories  under  ~roger.  For  details 
of  the  test  cases  see  the  report: 

AEA/TYKB/28006/TN/2 :  "Demonstration  Calculations  using  the 

ElectroMagnetic  PIC  Software  Suite  3DPIC" 

by  Eastwood,  Arter,  Brealey  and  Hockney,  September  1995. 

Each  test  case  has  a  directory  (ref)  for  the  single  processor  reference 
results  produced  with  the  single  processor  code  on  a  SUN  workstation 
at  Culham,  and  three  directories  (l-proc,  2-proc  and  8-proc)  for  results 
from  the  MIMD  parallel  PIC3D  code  produced  at  Maui.  These  are  for  the 
MIMD  code  running  respectively  on  1,  2  and  8  processors  (or  nodes) .  The 
maximum  number  of  processors  that  can  be  used  is  equal  to  the  number  of 
blocks,  so  this  is  also  given  below: 

4.3)  Wave  Test  Case,  uneSl,  with  10  timesteps  per  period  (8  blocks), 
-  for  100  steps: 

-hockney/MAUI-3DPIC/PIC3D/UNE51/Datum/ref 

/l-proc 

/2-proc 

/8-proc 

4.4)  Wave  Test  Case,  uneSl,  with  100  timesteps  per  period  for  1000  steps. 
-  This  was  setup  to  see  the  effect  of  sampling 

errors  on  the  wave: 

-hoc)aiey/MAUI-3DPIC/PIC3D/UNE51/SmallDT/ref 

/l-proc 

/2-proc 

/8-proc 

4.5)  Single  Particle  Orbit  Test,  porb21  (8  blocks)  for  400  steps: 


-hockney/MAUI-3DPIC/PIC3D/PORB21/Datum/ref 

/l-proc 

/2-proc 

/8-proc 

4.6)  Full  MILO  Simulation  (72  blocks)  for  2000  steps: 


-hockney/MAUI-3DPIC/PIC3D/MLS407/Datum/ref 

/l-proc 

/2-proc 

/8-proc 


5)  RUNNING  MIMD  PIC3D 


After  starting  PVM  as  described  above,  PIC3D  can  be  run  in  various  ways, 
but  I  have  adopted  the  following  procedure.  Starting  from  the  -hockney 
top  directory: 


%  rm  uneSl* 
%  rm  fort* 


Remove  leftover  files 
from  the  top  directory. 


%  cd  MAUI*/PIC3D/UNE51/Dattim/2-proc 


Go  to  test  directory 


%  cp  uneSl . dat .p2 .datumDT  input.dat  Copy  input  data  to  input.dat 

The  current  MIMD  version  for 
various  reasons  has  the  input 
file  wired  in  as  'input.dat'. 
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This  can  be  changed  later. 

Remove  previous  results. 
Failure  to  remove  old  *.tsd 
files  will  stop  the  program 
from  running. 

BUT  LEAVE  any  *.dat*  for 
future  use. 

PIC3D  should  run  taking  its 
data  from  'input.dat' 

Some  test  printouts  are  still  in  the  program,  but  the  output  should  look 
something  like  the  following,  with  a  write  to  the  screen  every  10  steps 
to  show  progress.  The  pvmfjoin  error  does  not  seem  to  matter: 

MAIN.F  started 

Enter  MCDPVM3/RHSPAW:  K=  -1 

Befor  NODASG(O)  MYPRES  =  1  NPRES=  2 

Enter  MCDPVM3 /RHSPAW:  K=  0 

After  pvmfspawn:  ntids(l)=  262151  inumt=  1 
libpvm  [t40006]:  gs_getgstid { )  failed  to  start  group  server:  No  such  file 
ERROR  in  pvmfjoin,  RHBARR,  inum  =  -14  MYPRES  =  1 
After  NODASG{0)  MYPRES  =  1  NPRES=  2 


Parallel  3DPIC 
PVM3  Version 
August,  1995 


3D  E/M  PIC  Simulation 
Eastwood,  Arter,  Brealey  and  Hockney 
AEA  Technology,  Culham  Laboratory 
Comput .  Phys.  Comm.  87,  155-178  (1995) 


Ntimber  of  Processors  =  2 
Active  MYPRES  =  1  NPRES=  2 
CNOUT  =une51.outp2:l 
NLRES  =  F  MYPRES  =  1 


Beg 

Step= 

70  mypres= 

1  npres= 

2 

Beg 

Step= 

80  mypres= 

1  npres= 

2 

Beg 

Step= 

90  mypres= 

1  npres= 

2 

Beg 

Step= 

100  mypres= 

1  npres= 

2 

Elapsed  time  this  run  =  2.680000  sec 

or  =  .044667  min 

or  =  .000744  hour 


Output  files  from  processor  with  MYPRES=1  will  appear  in  the  present 
test  directory,  but  output  from  all  other  processors  (MYPRES=2 , . . . ,NPRES) 
will  be  found  in  the  top  directory.  To  put  all  output  files  in  the  test 
directory  do  a  move: 

%  mv  -/uneSl*  .  Allow  uneSl.grd  from  top  directory  to 

overwrite  that  in  test  directory.  It 
contains  nothing  of  interest.  We  should 
give  the  *.grd  files  different  names  from 
each  processor,  to  stop  this  overwrite. 

6)  RETURNING  RESULTS  TO  CULHAM 


The  graphical  output  files  can  best  be  examined  at  Culham  using  the 
MPICTIM  and  ghost  graphics.  The  above  file  structure  is  repeated  at 
Culham  under  -roger  instead  of  -hocScney,  so  the  procedure  is  to  copy 
all  files  under  the  Maui  test  directory  to  the  same  directory  at  Culham. 
This  is  done  using  'ftp'  from  Culham  through  the  Harwell  gateway  as 
follows : 


MYPRES  =  1 


rfesunc%  cd 


rfesunc%  cd  MAUI*/PIC3D/UNE51/DatiJin/2-proc  Go  to  test  directory  at  Culham 
rfesunc%  ftp  gw  FTP  through  Harwell  gateway 

Connected  to  gw.aea.orgn.uk. 

220-Proxy  first  requires  authentication 

220  gw  FTP  proxy  (Version  VI . 3 )  ready. 

Name  (gw:roger):  rhockney 

331  Enter  authentication  password  for  rhockney 
Password:  <PWD> 

230  User  authenticated  to  proxy 

ftp>  user  hoc]cney@tsunami .  sp2  .mhpcc .  edu 

331-( - GATEWAY  CONNECTED  TO  tsunami.sp2.iiihpcc.edu - ) 

331-(220  fr3n02.mhpcc.edu  FTP  server  (Version  4.14  Fri  Aug  5  13:39:22  CDT  1994)  ready.) 
331  Password  required  for  hockney. 

Password:  <PWD> 

230  User  hoc)cney  logged  in. 

ftp>  binary  Don't  forget  BINARY  mode  for 

graphics  files 

200  Type  set  to  I. 

ftp>  hash  Hash  to  follow  progress 

Hash  mark  printing  on  (8192  bytes/hash  mark) . 

ftp>  cd  MAUI-3DPIC/PIC3D/UNE51/Datum/2-proc 

250  CWD  command  successful. 

ftp>  mget  * 

mget  input.dat?  y 

200  PORT  command  successful. 

150  Opening  data  connection  for  input.dat  (97210  bytes). 

############ 

226  Transfer  complete. 

local :  input . dat  remote :  input . dat 

97210  bytes  received  in  71  seconds  (1.3  Kbytes/s) 

mget  une51-01-0001 . tsd?  y 

200  PORT  command  successful. 

150  Opening  data  connection  for  une51-01-0001 . tsd  (4512  bytes). 

# 

226  Transfer  complete. 

local:  \ine51-01-0001 . tsd  remote:  une51-01-0001 . tsd 
4512  bytes  received  in  4 . 6  seconds  (0.96  Kbytes/s) 
mget  une51-01-0002 . tsd?  y 

mget  une51-02-0001 . tsd?  y 

mget  une51-02-0002 . tsd?  y 

mget  une51-03-0001 . tsd?  y 

mget  une51-03-0002 . tsd?  y 

mget  uneSl . dat .p2 .datumDT?  n 
mget  une51.grd?  y 

mget  une51 . outp2 ; 1?  y 

mget  une51 . outp2 : 2 ?  y 

ftp>  quit 

221  Goodbye . 
rfesunc% 


7)  RECOMPILATION 


After  any  changes  to  the  source,  the  program  should  be  recompiled 
to  make  a  new  executable,  pic3d,  in  the  usual  way.  This  should  be 
done  in  the  PIC3D  directory,  as  follows.  The  example  shows  a  recompile 
of  auxval.f  and  gstplt.f  followed  by  relinking: 

%  cd  MAUI*/PIC3D 
%  make 

til  -0  -qcharlen=2000  -c  auxval.f  Recompile 

**  auxval  ===  End  of  Compilation  1  === 

1501-510  Compilation  successful  for  file  auxval.f. 
cd  ../DIAG;  make  libdiag.a; 

til  -O  -qcharlen=2000  -c  gstplt.f  Recompile 

*•  gstplt  ===  End  of  Compilation  1  === 

1501-510  Compilation  successful  for  file  gstplt.f. 
ar  r  libdiag.a  gstplt.o 
touch  libdiag.a 
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cd  ../NETGLB;  make  libnetglb.a; 

Target  libnetglb.a  is  up  to  date. 

cd  ../NETBLK;  make  libnetblk.a; 

Target  libnetblk.a  is  up  to  date. 

cd  ../PARGLB;  make  libparglb.a; 

Target  libparglb.a  is  up  to  date. 

cd  ../PARBLK;  make  libparblk. a; 

Target  libparblk. a  is  up  to  date. 

cd  ../EMGLB;  make  libemglb.a; 

Target  libemglb.a  is  up  to  date. 

cd  ../EMBLK;  make  libemblk.a; 

Target  libemblk.a  is  up  to  date. 

cd  ../PEGLIB;  make  libpeglib.a; 

Target  libpeglib.a  is  up  to  date. 

cd  ../GLBLIB;  make  libglblib. a; 

Target  libglblib. a  is  up  to  date. 

cd  ../MCDPVM3;  make  libmcdlib.a; 

Target  libmcdlib.a  is  up  to  date. 

cd  ../CRONUS;  make  libcronus . a; 

Target  libcronus. a  is  up  to  date. 

cd  ../OLYPAR;  make  libolypar .a; 

Target  libolypar. a  is  up  to  date. 

cd  ../OLYMPUS;  make  libolympus . a; 
Target  libolympus. a  is  up  to  date. 

f77  -O  -qcharlen=2000  -o  pic3d  main.o 
filtr.o  cotrol.o  ctdepc.o  curode.o  ddot.o 
ndrun.o  expt.o  helmh.o  inital.o  labrun.o 
estrt.o  retb.o  stepbn.o  strage.o  tesend.o 
/EMGLB  -L.. /EMBLK  -L.. /PEGLIB  -L.. /GLBLIB 
g  -Inetglb  -Inetblk  -Iparglb  -Iparblk 
ronus  -L.. /OLYPAR  -lolypar  -lolympus  -Ighos 
3  Relink 


advb . o 

auxval.o 

bainit . o 

binit . 

o  btoh.o 

bufunl . 0 

ccinit . o 

ddot2 . o 

dfiltr .o 

dtoe. 0 

ebext . 0 

einit .  o 

embedO . o 

embed2 . o 

mindat . o 

movel . o 

move 2 . o 

newp . o 

output . o 

painit . o 

restor.o 

-L../DIAG  -L.. /NETGLB  -L.. /NETBLK  -L.. /PARGLB  -L.. /PARBLK  -L. 
-L../MCDPVM3  -L.. /CRONUS  -L.. /OLYMPUS  -L ../.. /ghost/lib  -Idi 
-lemglb  -lemglb  -lemblk  -Ipeglib  -Iglblib  -Imcdlib  -1 
t  -Igrid  -L/source/pd/pvm3 . 2 . 6/pvm3/lib/RS6K  -lfpvm3  -lpvm3  -Igpv 


%  mv  pic3d  “/pvm3/bin/RS6K 

overwrite  /u/hockney/pvm3/bin/RS6K/pic3d?  y 
% 


Move  new  executable  to  correct 
PVM  directory,  for  use  by 
pvmf spawn. 


END  README-MIMD  FILE 
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File :  -/hockney/SP2 /LPM3 / test/README-LPM3 


README-LPM3 


LPM3  Benchmark 


Roger  Hockney 
September  1995 


(1)  RUNNING  LPM3 


To  run  the  LPM3  Benchmark,  type  at  the  UNIX  prompt  (%)  on  a  node 
of  the  Maui  SP2 : 


%  cd  ~hockney/pvm3 /bin/RS6K 
%  cp  lpm3 .pvm3 . 3Aug95  lpm3 

%  cd  -hockney/SP2/LPM3 /test 
%  cp  neut50.dat  input.dat 


Go  to  pvm3  executable  directory 

Copy  executable  to  name  expected  by 
lpm3  code 

Go  to  directory  to  be  used  for  test 

Copy  data  file  to  working  data  file. 
We  do  the  8-block  test  as  an  example. 
Other  data  files  are: 


neutSO . dat 
neutSl . dat 
neut52 .dat 
neut53 .dat 

%  vi  input . dat 


%  pvm 

pvin>  add 
pvm>  add 
pvm>  add 

pvm>  quit 

%  lpm3 


8-block  test 
64-block  test 
512-block  test 
4096-block  test 


Edit  2nd  line  to  number  of 
processors  as  required,  up  to  the  a 
maximum  equal  to  the  number  of  blocks . 

Start  a  PVM  machine  and  add  as  many 
processors  as  required. 


Quit  PVM  Console 
Start  lpm3 


All  the  important  output  goes  to  the  screen  with  one  line  printed 
every  10  steps.  The  timing  results  are  given  at  the  end  in  the 
***★★★*****★★*★**★*★****★*★*★★ 

LPM3  -  RESULTS  SUMMARY 

★  *★★*★★*★★★★*★*♦★★*★★★**★**’*★* 

The  key  benchmark  numbers  are,  for  example: 


Number  processors,  NPRES  =  2 

Elapsed  Benchmark  wall  time  =  55.62006760  s 


Temporal  Perforance  =  1.797912240  tstep/s 


This  output  is  repeated  at  the  end  of  the  output  from  the  first  processor 
in  file: 

O_neut50p2 : 1 

The  second  processor's  output  file  appears  in  the  top  directory 

%  mv  ~/o_*  .  Move  output  from  other  processors 

to  test  directory 

(2)  SCREEN  OUTPUT 


The  screen  output  for  the  case  neut50.dat  {8  blocks)  is  in  file 

lpm3 . out 

and  follows.  The  initial  ERROR  does  not  seem  to  matter: 

ERROR  in  pvmf join,  RHBARR,  inum  =  -14  MYPRES  =  1 


README-LPM3 


LPM3  BENCHMARK 


3D  Periodic  PIC  Plasma 
Roger  Hockney,  July  1994 


Number  of  Processors  =  2 

NSTEP=  0  MYPRES=  1  :  ***  PBLOCK,  cls6b.f:  MAXBLK  OK  ** 

IBLMAX=  8  .LE.  MAXBLK=  5096 

NSTEP=  0  MYPRES=  1  :  ***  PBLOCK,  cls6b.f:  MAXPCH  OK  ** 

IPAMAX=  72  .LE.  MAXPCH  40000 


Random  Number  test:  first  20 


NSEED=  0 
RANDF(0)= 
RANDF ( 0 )  = 
RANDF  ( 0 )  = 
RANDF { 0 ) = 
RANDF { 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF ( 0 ) = 
RANDF { 0 ) = 
RANDF { 0 ) = 
RANDF { 0 ) = 
RANDF { 0 ) = 
RANDF ( 0 ) = 
RANDF  ( 0 )  = 


NSW=  0 

0.1809234619 

0.3858184814 

0.6827545166 

0.6078643799 

0.5761871338 

0.5847930908 

0.4784088135 

0.2754211426E- 

0.6910705566E- 

0.9595489502 

0.5904693604 

0.2167510986 

0.3471832275 

0.9475860596 

0.2064361572 

0.1129913330 

0.9791564941E 

0.9864044189 

0.5138092041 


NSEED=  11857 
NSEED=  25285 
NSEED=  44745 
NSEED=  39837 
NSEED=  37761 
NSEED=  38325 
NSEED=  31353 
■01  NSEED=  1805 
•01  NSEED=  4529 
NSEED=  62885 
NSEED=  38697 
NSEED=  14205 
NSEED=  22753 
NSEED=  62101 
NSEED=  13529 
NSEED=  7405 
■01  NSEED=  6417 
NSEED=  64645 
NSEED=  33673 


average=  0.4990450740  of  10000  randoms 

NSTEP=  0  MYPRES=  1  :  ***  PINIT,  clsl3.f:  MDPART  OK  *** 

IMAXPT=  21504  .LE.  MDPART=  520000 

NSTEP=  0  MyPRES=  1  :  ***  PINIT,  clsl3.f:  MDCOA  OK  *** 

INDLPA  73  .LE.  MDCOA=  82000 
*■*•★★'*•****♦★**★*♦★■*★*'*’*****★★********★*** 

LOAD  BALANCE  ANALYSIS 

★  ★★★★Tt********************************** 


2  Processes  each  have  4  blocks 

TOTALS 


Processes  2  should  be  2 

Blocks  8  should  be  8 


Timer  Overhead,  TO  =  0 . 151040000000435271E-04 


=== 

LPM3  BENCHMARK 

=== 

It  II 

11  11 

II  11 

UKAEA-Culham/USAF  E/M-PIC  Benchmark 

=== 

=== 

3D  periodic  cubical  electron  Plasma 

=  =  = 

)  11  11 

1  II  11 

1  II  II 

Program 

:  LPM3 :  Local  Partical  Mesh 

=== 

=== 

Version 

:  PVM  +  Fortran  77 

=  =  = 

=== 

Author 

:  Roger  W.  Hockney 

=  =  = 

=== 

Code  Opt 

:  Roger  W.  Hockney 

=  =  = 

=== 

Update 

:  Oct,  1994;  Release;  1.0 

=== 

--- 

=  =  = 

Run  on  400  node  IBM  SP2  at  USAF/DOD 
Maui  High  Performance  Computer  Center 


Using  thin  nodes  with  64MByte  memory 


Benchmarker  was  Roger  Hockney 


The  calculation  proceeds  ...Please,  wait. 


Benchmark  run  started,  step  =  0 


Beg 

Step 

= 

1 

mypres 

= 

1 

parts 

= 

0 

mesh 

0 

nseed 

= 

15953 

itot= 

0 

ileave= 

0 

Beg 

Step 

= 

2 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

55 

Beg 

Step 

= 

3 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

i leaver 

71 

Beg 

Step 

= 

4 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

50 

Beg 

Step 

= 

5 

mypres 

= 

1 

parts 

= 

2048 

mesh 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

53 

Beg 

Step 

= 

6 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

58 

Beg 

Step 

= 

7 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

61 

Beg 

Step 

= 

8 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

55 

Beg 

Step 

= 

9 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

74 

Beg 

Step 

= 

10 

mypres 

= 

1 

parts 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

52 

Beg 

Step 

20 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

53 

Beg 

Step 

= 

30 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

55 

Beg 

Step 

= 

40 

mypres 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

62 

Beg 

Step 

= 

50 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

46 

Beg 

Step 

= 

60 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

59 

Beg 

Step 

= 

70 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

58 

Beg 

Step 

= 

80 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

53 

Beg 

Step 

= 

90 

mypres 

= 

1 

parts 

= 

2047 

mesh 

= 

960 

nseed 

15953 

itot= 

512 

ileave= 

54 

Beg 

Step 

= 

100 

mypres 

= 

1 

parts 

= 

2048 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

511 

ileave= 

64 

End 

Step 

= 

100 

mypres 

= 

1 

parts 

= 

2051 

mesh 

= 

960 

nseed 

= 

15953 

itot= 

512 

ileave= 

53 

Benchmark  run  ended,  step  =  100 

**************************************** 

LPM3  -  VALIDATION  CHECK 
***************************************** 

Total  number  particles  all  processes: 

this  run  =  4096 
Reference  number  is  =  4096 

percent  difference  =  0 . OOOOOOOOOOE+00  % 

Total  number  mesh-points  all  processes: 

this  run  =  1920 
Reference  number  is  =  1920 


★*****★****★***★★★*******★***♦*★*★****** 

DIFFERENCE  <  10%,  VALIDATION  OK 
★  ***★★■*•★***★★***★**★******♦****★**★***** 

**************************************** 

LPM3  -  RESULT  SUMMARY 

Hr*************************************** 

Number  processes,  NPRES  =  2 

Number  blocks ,  NBLOCK  =  8 
Number  elements,  INELEM  =  512 
Number  patches,  NPATCH  =  72 

Input  file,  CHREFN  =neut50 

Output  files  =o_neut50p2 : <n>  for  proc  <n> 
MXPASW=2 :  per-processor  code 

Communications  analysis: 

Elapsed  (XPATCH)  wall  time  =  7.802058697  s 

(  17.04426765  %) 

Total  number  CALLS  to  XPATCH  =  800 
Total  number  exchanges (CSEND+CRECV)  =  800 
Total  byte  CSEND  =  725200  Byte 
Total  byte  CRECV  =  725308  Byte 
Average  CSEND  length  =  906.5000000  Byte 
Average  CRECV  length  =  906.6350098  Byte 
Time  per  exchange  =  9752.573242  us 
CSEND  bandwidth  =  0 . 9294982255E-01  MByte/s 

Number  of  Timesteps  this  run  =  100 
Elapsed  Benchmark  wall  time  =  45.77526474  s 


Temporal  Performance  =  2.184586048  tstep/s 


README-LPM3 

**************************************** 

**♦*******★★★***★********★*★**★******•*** 

LOAD  BALANCE  ANALYSIS 
*•*****★*♦★**★**★*★*******★*****★****★*•** 

2  Processes  each  have  4  blocks 

TOTALS 


Processes  2  should  be  2 

Blocks  8  should  be  8 

★  ★★★★★★★★★★It********************"******** 

Elapsed  time  this  run  =  46.907288  sec 

or  =  .781788  min 

or  =  .013030  hour 


